Nov 04 01:54:51.154621 ip-10-0-134-204 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 01:54:51.573978 ip-10-0-134-204 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:54:51.573978 ip-10-0-134-204 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 01:54:51.573978 ip-10-0-134-204 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:54:51.573978 ip-10-0-134-204 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:54:51.580824 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580740    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:54:51.580824 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580820    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:54:51.580824 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580825    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:54:51.580824 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580828    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:54:51.580824 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580831    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580834    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580837    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580840    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580846    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580849    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580852    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580854    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580857    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580860    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580863    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580870    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580873    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580876    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580878    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580881    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580883    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580889    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580892    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580895    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:54:51.581005 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580897    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580900    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580903    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580905    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580908    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580911    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580913    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580916    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580920    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580924    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580930    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580932    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580935    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580938    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580941    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580944    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580947    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580950    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580953    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580955    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:54:51.581502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580958    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580961    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580966    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580969    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580974    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580978    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580981    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580984    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580987    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580989    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580992    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580994    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.580998    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581001    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581006    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581010    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581012    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581015    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581018    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:54:51.581978 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581022    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581024    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581027    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581030    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581033    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581036    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581039    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581042    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581047    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581050    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581053    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581056    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581058    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581061    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581064    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581066    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581069    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581072    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581075    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:54:51.582432 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581078    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581083    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581086    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581088    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581091    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581094    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581097    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581099    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581102    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581105    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581107    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581110    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581113    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581115    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581120    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581123    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581126    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581129    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581131    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581134    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:54:51.582882 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581137    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:54:51.583401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581139    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:54:51.583401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581142    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:54:51.583401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.581146    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:54:51.583938 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583925    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:54:51.583938 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583937    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:54:51.583988 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583941    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:54:51.583988 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583944    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:54:51.583988 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583948    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:54:51.583988 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583954    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:54:51.583988 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583957    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:54:51.583988 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583959    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:54:51.583988 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583963    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:54:51.583988 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583966    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:54:51.583988 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583970    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:54:51.583988 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583975    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:54:51.583988 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583978    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:54:51.583988 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583981    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:54:51.583988 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583984    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:54:51.583988 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583988    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:54:51.583988 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583991    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.583997    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584000    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584003    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584005    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584008    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584011    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584013    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584016    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584020    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584023    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584026    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584029    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584034    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584037    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584039    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584042    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584044    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584047    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:54:51.584453 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584050    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584052    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584055    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584058    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584060    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584063    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584068    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584071    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584074    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584076    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584079    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584082    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584084    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584087    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584090    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584094    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584096    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584099    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584102    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584107    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:54:51.584906 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584109    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584112    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584114    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584117    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584119    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584122    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584125    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584128    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584130    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584133    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584136    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584138    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584143    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584146    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584149    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584153    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584157    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584160    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584162    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584165    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:54:51.585401 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584168    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584171    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584174    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584176    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584181    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584185    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584188    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584192    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584195    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584197    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584200    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584203    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584205    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584208    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584211    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584213    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584218    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584221    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584224    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:54:51.585874 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584226    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584229    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584232    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584234    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584237    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584239    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584242    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584245    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584247    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584250    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584255    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.584258    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585044    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585050    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585053    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585057    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585060    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585063    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585066    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585073    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:54:51.586351 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585075    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585078    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585081    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585083    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585086    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585089    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585092    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585094    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585097    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585099    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585102    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585105    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585110    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585113    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585115    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585118    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585121    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585124    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585127    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:54:51.586823 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585129    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585132    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585135    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585137    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585140    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585145    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585149    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585152    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585154    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585157    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585160    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585163    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585166    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585169    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585171    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585174    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585177    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585179    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585184    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:54:51.587273 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585187    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585189    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585192    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585195    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585197    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585200    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585202    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585205    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585208    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585211    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585214    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585216    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585231    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585234    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585238    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585240    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585243    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585246    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585252    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585255    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:54:51.587745 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585258    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585261    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585263    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585266    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585272    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585276    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585294    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585297    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585300    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585303    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585305    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585308    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585311    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585314    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585316    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585319    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585324    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585327    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585330    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:54:51.588248 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585332    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585335    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585338    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585341    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585343    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585346    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585349    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585351    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585354    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585356    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585361    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585365    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585368    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585371    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585374    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585377    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585380    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585382    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585385    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585390    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:54:51.588716 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.585485    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586673    2574 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586686    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586694    2574 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586699    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586704    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586707    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586711    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586716    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586719    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586722    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586726    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586729    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586733    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586741    2574 flags.go:64] FLAG: --cgroup-root=""
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586751    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586754    2574 flags.go:64] FLAG: --client-ca-file=""
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586757    2574 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586760    2574 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586765    2574 flags.go:64] FLAG: --cluster-domain=""
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586768    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586771    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586774    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586777    2574 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 01:54:51.589185 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586785    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586788    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586792    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586795    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586799    2574 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586802    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586805    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586808    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586812    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586817    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586820    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586823    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586826    2574 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586830    2574 flags.go:64] FLAG: --enable-server="true"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586833    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586837    2574 flags.go:64] FLAG: --event-burst="100"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586840    2574 flags.go:64] FLAG: --event-qps="50"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586843    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586846    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586849    2574 flags.go:64] FLAG: --eviction-hard=""
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586854    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586857    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586860    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586863    2574 flags.go:64] FLAG: --eviction-soft=""
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586866    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 01:54:51.589779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586869    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586872    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586875    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586878    2574 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586881    2574 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586884    2574 flags.go:64] FLAG: --feature-gates=""
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586888    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586891    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586894    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586897    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586900    2574 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586903    2574 flags.go:64] FLAG: --help="false"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586906    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586909    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586913    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586916    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586920    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586924    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586927    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586930    2574 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586932    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586935    2574 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586938    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 01:54:51.590417 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586941    2574 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586944    2574 flags.go:64] FLAG: --kube-reserved=""
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586947    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586950    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586953    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586956    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586960    2574 flags.go:64] FLAG: --lock-file=""
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586962    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586965    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586968    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586974    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586977    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586980    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586983    2574 flags.go:64] FLAG: --logging-format="text"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586986    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586989    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586992    2574 flags.go:64] FLAG: --manifest-url=""
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586995    2574 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.586999    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587002    2574 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587006    2574 flags.go:64] FLAG: --max-pods="110"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587009    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587012    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587015    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587018    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 01:54:51.590989 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587021    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587024    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587027    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587035    2574 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587038    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587041    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587044    2574 flags.go:64] FLAG: --pod-cidr=""
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587047    2574 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587050    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587053    2574 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587056    2574 flags.go:64] FLAG: --port="10250"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587059    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587062    2574 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-003e72c3116aa78d7"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587065    2574 flags.go:64] FLAG: --qos-reserved=""
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587068    2574 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587071    2574 flags.go:64] FLAG: --register-node="true"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587074    2574 flags.go:64] FLAG: --register-with-taints=""
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587079    2574 flags.go:64] FLAG: --registry-burst="10"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587082    2574 flags.go:64] FLAG: --registry-qps="5"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587084    2574 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587087    2574 flags.go:64] FLAG: --reserved-memory=""
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587091    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587094    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587097    2574 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587100    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587102    2574 flags.go:64] FLAG: --runonce="false"
Nov 04 01:54:51.591663 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587105    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587108    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587112    2574 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587115    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587117    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587121    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587125    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587128    2574 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587131    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587134    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587136    2574 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587139    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587142    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587145    2574 flags.go:64] FLAG: --system-cgroups=""
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587148    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587154    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587157    2574 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587160    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587165    2574 flags.go:64] FLAG: --tls-min-version=""
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587168    2574 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587170    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587173    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587180    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587184    2574 flags.go:64] FLAG: --v="2"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587188    2574 flags.go:64] FLAG: --version="false"
Nov 04 01:54:51.592298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587192    2574 flags.go:64] FLAG: --vmodule=""
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587196    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587199    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587360    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587365    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587369    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587373    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587376    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587379    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587382    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587385    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587387    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587392    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587395    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587399    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587401    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587404    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587408    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587411    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:54:51.592899 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587413    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587417    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587421    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587424    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587426    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587429    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587432    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587434    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587437    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587440    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587444    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587447    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587449    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587452    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587455    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587457    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587460    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587462    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587465    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:54:51.593379 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587468    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587470    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587473    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587475    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587478    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587480    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587484    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587487    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587490    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587493    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587495    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587498    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587501    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587503    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587506    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587508    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587511    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587513    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587516    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:54:51.593865 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587518    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587523    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587527    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587530    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587535    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587538    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587541    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587544    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587547    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587549    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587552    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587555    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587558    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587560    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587563    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587566    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587568    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587571    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587574    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:54:51.594345 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587578    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587580    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587583    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587586    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587589    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587591    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587594    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587597    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587599    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587602    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587604    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587607    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587610    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587612    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587614    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587617    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587620    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587624    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587627    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587630    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:54:51.594806 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587632    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:54:51.595297 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587635    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:54:51.595297 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587637    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:54:51.595297 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587640    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:54:51.595297 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587643    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:54:51.595297 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587645    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:54:51.595297 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587648    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:54:51.595297 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587651    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:54:51.595297 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587653    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:54:51.595297 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587656    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:54:51.595297 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587658    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:54:51.595297 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587660    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:54:51.595297 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587665    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:54:51.595297 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.587691    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:54:51.595297 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587731    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:54:51.595297 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587735    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:54:51.595297 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587739    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587742    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587745    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587748    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587751    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587754    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587756    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587759    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587762    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587764    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587767    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587770    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587772    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587775    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587779    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587781    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587784    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587787    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587789    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:54:51.595678 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587792    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587796    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587801    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587804    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587807    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587811    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587814    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587817    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587820    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587824    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587827    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587829    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587832    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587834    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587845    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587848    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587851    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587853    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587856    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:54:51.596122 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587858    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587861    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587863    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587866    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587869    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587871    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587874    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587876    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587880    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587883    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587886    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587888    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587891    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587894    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587896    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587899    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587901    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587904    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587906    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587909    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:54:51.596636 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587912    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587915    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587918    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587921    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587924    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587926    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587929    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587932    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587935    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587937    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587940    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587943    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587945    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587948    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587950    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587953    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587955    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587958    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587960    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587963    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:54:51.597095 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587967    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587969    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587972    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587974    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587977    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587980    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587982    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587986    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587989    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587991    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587995    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.587998    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.588001    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.588004    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.588008    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.588011    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.588013    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.588016    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.588019    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:54:51.597615 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.588021    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.588024    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.588026    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.588030    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.588033    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.588035    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.588038    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.595110    2574 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.595125    2574 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.595139    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.595144    2574 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595199    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595204    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595207    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595211    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595214    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595216    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595219    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595222    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:54:51.598057 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595225    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595227    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595230    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595233    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595235    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595238    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595241    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595243    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595246    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595249    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595251    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595254    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595257    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595259    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595262    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595265    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595267    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595270    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595272    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:54:51.598520 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595275    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595292    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595304    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595307    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595310    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595312    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595315    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595319    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595323    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595326    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595329    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595332    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595335    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595337    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595340    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595342    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595345    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595347    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595350    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:54:51.598980 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595352    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595355    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595358    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595361    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595364    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595368    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595371    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595374    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595376    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595379    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595381    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595384    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595386    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595389    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595391    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595394    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595397    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595399    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595402    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595405    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:54:51.599421 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595407    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595410    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595412    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595415    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595418    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595420    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595423    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595425    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595427    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595430    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595432    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595435    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595437    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595440    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595443    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595446    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595448    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595451    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595453    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595456    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:54:51.599907 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595459    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595461    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595464    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595466    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595469    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595472    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595474    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595477    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595479    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595483    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595485    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595488    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595490    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595493    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595495    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595498    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595500    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595502    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595505    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:54:51.600396 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595507    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:54:51.600840 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.595535    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:54:51.600840 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595674    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:54:51.600840 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595678    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:54:51.600840 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595681    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:54:51.600840 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595684    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:54:51.600840 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595687    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:54:51.600840 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595689    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:54:51.600840 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595692    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:54:51.600840 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595695    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:54:51.600840 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595697    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:54:51.600840 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595700    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:54:51.600840 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595703    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:54:51.600840 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595705    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:54:51.600840 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595708    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:54:51.600840 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595710    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:54:51.600840 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595713    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595716    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595718    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595720    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595724    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595726    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595729    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595732    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595735    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595737    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595739    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595742    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595744    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595747    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595750    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595752    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595754    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595757    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595759    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:54:51.601203 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595762    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595764    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595767    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595769    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595772    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595775    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595777    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595780    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595782    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595785    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595787    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595790    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595793    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595796    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595798    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595801    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595803    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595806    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595808    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:54:51.601741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595811    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595813    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595816    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595819    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595821    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595824    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595826    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595829    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595831    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595834    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595836    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595839    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595841    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595845    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595848    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595851    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595854    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595856    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595858    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595861    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:54:51.602188 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595864    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595866    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595869    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595872    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595874    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595877    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595880    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595882    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595886    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595890    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595892    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595895    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595898    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595900    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595903    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595906    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595908    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595911    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595913    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:54:51.602693 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595916    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:54:51.603124 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595918    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:54:51.603124 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595921    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:54:51.603124 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595924    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:54:51.603124 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595926    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:54:51.603124 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595928    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:54:51.603124 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595931    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:54:51.603124 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595933    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:54:51.603124 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595936    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:54:51.603124 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595938    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:54:51.603124 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595941    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:54:51.603124 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595943    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:54:51.603124 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595946    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:54:51.603124 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595949    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:54:51.603124 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.595951    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:54:51.603124 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.595976    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:54:51.603124 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596008    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596013    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596016    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596019    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596021    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596024    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596027    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596029    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596032    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596035    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596037    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596040    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596043    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596046    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596048    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596051    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596053    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596056    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596058    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:54:51.603508 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596060    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596063    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596066    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596068    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596071    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596073    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596076    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596078    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596081    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596084    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596086    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596089    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596092    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596094    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596096    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596099    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596101    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596104    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596107    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:54:51.603965 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596110    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596112    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596115    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596118    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596120    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596123    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596125    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596131    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596134    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596137    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596140    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596142    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596146    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596149    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596152    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596155    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596157    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596160    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596163    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596167    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:54:51.604414 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596170    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596173    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596176    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596178    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596181    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596183    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596186    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596189    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596191    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596193    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596196    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596199    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596201    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596206    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596209    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596213    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596215    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596218    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596220    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:54:51.605596 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596224    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596226    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596230    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596232    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596235    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596239    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596242    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596244    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596247    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596249    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596252    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596254    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596257    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596259    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596263    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596266    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596268    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596271    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596273    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596276    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:54:51.606165 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596293    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:54:51.606680 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596297    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:54:51.606680 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596300    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:54:51.606680 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596302    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:54:51.606680 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596305    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:54:51.606680 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596307    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:54:51.606680 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596310    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:54:51.606680 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596313    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:54:51.606680 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:51.596315    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:54:51.606680 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.596371    2574 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 01:54:51.606680 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.598716    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 04 01:54:51.606680 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.599663    2574 server.go:1014] "Starting client certificate rotation"
Nov 04 01:54:51.606680 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.599761    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 01:54:51.606680 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.599796    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 01:54:51.606680 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.601800    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:54:51.606680 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.604592    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:54:51.616913 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.616897    2574 log.go:25] "Validated CRI v1 runtime API"
Nov 04 01:54:51.620734 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.620721    2574 log.go:25] "Validated CRI v1 image API"
Nov 04 01:54:51.623024 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.623001    2574 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 01:54:51.626334 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.626305    2574 fs.go:135] Filesystem UUIDs: map[4c658a12-fb8f-4245-a666-190348527387:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b30cc0a9-c0f6-4898-bae5-37d34deda149:/dev/nvme0n1p3]
Nov 04 01:54:51.626422 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.626331    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 04 01:54:51.632041 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.631933    2574 manager.go:217] Machine: {Timestamp:2026-11-04 01:54:51.630515759 +0000 UTC m=+0.352551211 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3115134 MemoryCapacity:33164324864 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ff0932fd495d05d5e309ac8fcf97e SystemUUID:ec2ff093-2fd4-95d0-5d5e-309ac8fcf97e BootID:2f49a374-6381-48f4-b347-391da6f491c3 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582160384 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:78:3c:10:b8:ab Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:78:3c:10:b8:ab Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:3b:28:79:b5:d3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164324864 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 01:54:51.632041 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.632031    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 01:54:51.632156 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.632108    2574 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 01:54:51.633310 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.633270    2574 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 01:54:51.633438 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.633312    2574 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-204.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 01:54:51.633480 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.633441    2574 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 01:54:51.633480 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.633449    2574 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 01:54:51.633480 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.633475    2574 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:54:51.633560 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.633491    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:54:51.634710 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.634700    2574 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 01:54:51.639108 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.639096    2574 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 01:54:51.639225 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.639216    2574 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 01:54:51.640237 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.640227    2574 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 01:54:51.640275 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.640244    2574 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 01:54:51.640275 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.640262    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 01:54:51.640275 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.640274    2574 kubelet.go:405] "Adding apiserver pod source"
Nov 04 01:54:51.640407 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.640298    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 01:54:51.640689 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.640675    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:54:51.642857 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.642840    2574 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 01:54:51.649212 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.649197    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:54:51.650106 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.650091    2574 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 01:54:51.650180 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.650132    2574 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 01:54:51.650347 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.650335    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 01:54:51.650411 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.650355    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 01:54:51.650411 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.650363    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 01:54:51.650411 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.650372    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 01:54:51.650411 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.650381    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 01:54:51.650411 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.650390    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 01:54:51.651036 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.651026    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 01:54:51.651087 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.651041    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 01:54:51.651087 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.651052    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 01:54:51.651087 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.651062    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 01:54:51.651087 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.651076    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 01:54:51.651227 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.651091    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 01:54:51.652746 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.652735    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 01:54:51.652807 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.652750    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 01:54:51.656504 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.656489    2574 server.go:1263] "Started kubelet"
Nov 04 01:54:51.656599 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.656569    2574 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 01:54:51.656704 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.656645    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 01:54:51.656750 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.656727    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 01:54:51.657199 ip-10-0-134-204 systemd[1]: Started Kubernetes Kubelet.
Nov 04 01:54:51.658267 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.658220    2574 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 01:54:51.660065 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.660054    2574 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 01:54:51.663542 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:51.663523    2574 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 04 01:54:51.666241 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.666224    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 01:54:51.666868 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.666850    2574 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 01:54:51.667027 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:51.667000    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:54:51.667660 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.667643    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 01:54:51.667660 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.667644    2574 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 01:54:51.667779 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.667670    2574 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 01:54:51.667815 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.667795    2574 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 01:54:51.667815 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.667803    2574 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 01:54:51.668069 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.668053    2574 factory.go:153] Registering CRI-O factory
Nov 04 01:54:51.668069 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.668070    2574 factory.go:223] Registration of the crio container factory successfully
Nov 04 01:54:51.668298 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.668265    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 01:54:51.668356 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.668299    2574 factory.go:55] Registering systemd factory
Nov 04 01:54:51.668356 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.668309    2574 factory.go:223] Registration of the systemd container factory successfully
Nov 04 01:54:51.668356 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.668328    2574 factory.go:103] Registering Raw factory
Nov 04 01:54:51.668356 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.668356    2574 manager.go:1199] Started watching for new ooms in manager
Nov 04 01:54:51.670051 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.670038    2574 manager.go:319] Starting recovery of all containers
Nov 04 01:54:51.682355 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.682337    2574 manager.go:324] Recovery completed
Nov 04 01:54:51.686071 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.686057    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:54:51.687624 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.687592    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:54:51.687695 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.687643    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:54:51.687695 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.687660    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:54:51.688222 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.688206    2574 cpu_manager.go:226] "Starting" policy="none"
Nov 04 01:54:51.688222 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.688220    2574 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 01:54:51.688401 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.688213    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:54:51.688401 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.688239    2574 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 01:54:51.689108 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.689089    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:54:51.689186 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.689125    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:54:51.689186 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.689142    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:54:51.690484 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.690468    2574 policy_none.go:50] "Start"
Nov 04 01:54:51.690527 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.690490    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 01:54:51.690527 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.690505    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 01:54:51.692539 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.692528    2574 policy_none.go:44] "Start"
Nov 04 01:54:51.744616 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.725845    2574 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 01:54:51.744616 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.730115    2574 manager.go:341] "Starting Device Plugin manager"
Nov 04 01:54:51.744616 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:51.730143    2574 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 01:54:51.744616 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.730151    2574 server.go:87] "Starting device plugin registration server"
Nov 04 01:54:51.744616 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.730419    2574 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 01:54:51.744616 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.730429    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 01:54:51.744616 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.730534    2574 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 01:54:51.744616 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.730618    2574 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 01:54:51.744616 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.730625    2574 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 01:54:51.744616 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:51.731203    2574 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 01:54:51.744616 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:51.731260    2574 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:54:51.744616 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.737440    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 01:54:51.744616 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.738528    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 01:54:51.744616 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.738544    2574 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 01:54:51.744616 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.738561    2574 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 01:54:51.744616 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:51.738592    2574 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 04 01:54:51.800731 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.800704    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-204.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 04 01:54:51.830778 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.830728    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:54:51.831496 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.831482    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:54:51.831571 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.831511    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:54:51.831571 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.831523    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:54:51.831571 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.831545    2574 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:51.839778 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.839642    2574 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal"]
Nov 04 01:54:51.839868 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.839814    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:54:51.840723 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.840703    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:54:51.840813 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.840733    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:54:51.840813 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.840744    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:54:51.843337 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.843325    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:54:51.843489 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.843472    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:51.843551 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.843518    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:54:51.844062 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.844042    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:54:51.844135 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.844072    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:54:51.844135 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.844087    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:54:51.844135 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.844095    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:54:51.844135 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.844113    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:54:51.844135 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.844125    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:54:51.846233 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.846214    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:51.846308 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.846253    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:54:51.846932 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.846918    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:54:51.847002 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.846942    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:54:51.847002 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.846964    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:54:51.848997 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:51.848971    2574 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-204.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 04 01:54:51.849790 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:51.848855    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-204.us-west-1.compute.internal.18e4b4c0c67346f9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-204.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-204.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 01:54:51.656464121 +0000 UTC m=+0.378499573,LastTimestamp:2026-11-04 01:54:51.656464121 +0000 UTC m=+0.378499573,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-204.us-west-1.compute.internal,}"
Nov 04 01:54:51.854745 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:51.854724    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-204.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 04 01:54:51.854745 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:51.854723    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 04 01:54:51.867933 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.867913    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6b3c9ba0af4012b95306d2f2cfa3c61-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"f6b3c9ba0af4012b95306d2f2cfa3c61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:51.868046 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.867938    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6b3c9ba0af4012b95306d2f2cfa3c61-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"f6b3c9ba0af4012b95306d2f2cfa3c61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:51.868046 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.867991    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/56dfafb26cd80e2f2f640fddc6c0403a-config\") pod \"kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"56dfafb26cd80e2f2f640fddc6c0403a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:51.869815 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:51.869797    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found" node="ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:51.871770 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:51.871752    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found" node="ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:51.879714 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.879699    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 01:54:51.879775 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.879750    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b8792"
Nov 04 01:54:51.964765 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.964734    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:54:51.965581 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.965565    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:54:51.968571 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.968555    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/56dfafb26cd80e2f2f640fddc6c0403a-config\") pod \"kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"56dfafb26cd80e2f2f640fddc6c0403a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:51.968650 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.968579    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6b3c9ba0af4012b95306d2f2cfa3c61-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"f6b3c9ba0af4012b95306d2f2cfa3c61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:51.968650 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.968594    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6b3c9ba0af4012b95306d2f2cfa3c61-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"f6b3c9ba0af4012b95306d2f2cfa3c61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:51.968650 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.968648    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6b3c9ba0af4012b95306d2f2cfa3c61-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"f6b3c9ba0af4012b95306d2f2cfa3c61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:51.968763 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.968662    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6b3c9ba0af4012b95306d2f2cfa3c61-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"f6b3c9ba0af4012b95306d2f2cfa3c61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:51.968763 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.968664    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/56dfafb26cd80e2f2f640fddc6c0403a-config\") pod \"kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"56dfafb26cd80e2f2f640fddc6c0403a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:51.971311 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:51.971296    2574 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:51.971386 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:51.971317    2574 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-204.us-west-1.compute.internal\": node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:54:52.087333 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:52.087262    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:54:52.174244 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:52.174218    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:52.177388 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:52.177367    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:52.188236 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:52.188215    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:54:52.289170 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:52.289147    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:54:52.390070 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:52.390004    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:54:52.490957 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:52.490933    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:54:52.591507 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:52.591478    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:54:52.599175 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:52.599153    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 04 01:54:52.599336 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:52.599319    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 01:54:52.599378 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:52.599350    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 01:54:52.599462 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:52.599366    2574 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://ab045e258af594899a140baaafc158e1-d72efd8280e6925c.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-134-204.us-west-1.compute.internal.18e4b4c0c84ec27c\": read tcp 10.0.134.204:36754->3.143.52.118:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-134-204.us-west-1.compute.internal.18e4b4c0c84ec27c  default   4778 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-204.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-204.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-204.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 01:54:51 +0000 UTC,LastTimestamp:2026-11-04 01:54:51.840719986 +0000 UTC m=+0.562755445,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-204.us-west-1.compute.internal,}"
Nov 04 01:54:52.692580 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:52.692530    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:54:52.794189 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:52.794160    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:54:52.885509 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:52.885477    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 01:49:52 +0000 UTC" deadline="2028-03-30 06:42:24.97559633 +0000 UTC"
Nov 04 01:54:52.885509 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:52.885506    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12292h47m32.090093957s"
Nov 04 01:54:52.894914 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:52.894892    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:54:52.904741 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:52.904724    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:54:52.905680 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:52.905664    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:54:52.905743 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:52.905698    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:54:52.905743 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:52.905716    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:54:52.995102 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:52.995031    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:54:53.091076 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.091052    2574 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:54:53.095348 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:53.095327    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:54:53.196998 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:53.196962    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:54:53.280438 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.280381    2574 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:54:53.367946 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.367913    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:53.427630 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:53.427599    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6b3c9ba0af4012b95306d2f2cfa3c61.slice/crio-c90dfcd721e4bf20cab1e57beb6127a78cb16c800014c20d17192be40af26ba5 WatchSource:0}: Error finding container c90dfcd721e4bf20cab1e57beb6127a78cb16c800014c20d17192be40af26ba5: Status 404 returned error can't find the container with id c90dfcd721e4bf20cab1e57beb6127a78cb16c800014c20d17192be40af26ba5
Nov 04 01:54:53.427910 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:53.427886    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56dfafb26cd80e2f2f640fddc6c0403a.slice/crio-a4cf7f99d75dbefad68a1a3ecfe1f4bc0aa617025acd7fe6bc3aa4552455a86a WatchSource:0}: Error finding container a4cf7f99d75dbefad68a1a3ecfe1f4bc0aa617025acd7fe6bc3aa4552455a86a: Status 404 returned error can't find the container with id a4cf7f99d75dbefad68a1a3ecfe1f4bc0aa617025acd7fe6bc3aa4552455a86a
Nov 04 01:54:53.433739 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.433723    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:54:53.437256 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.437195    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:54:53.438142 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.438129    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:54:53.501843 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.501825    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:54:53.558171 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.558119    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:54:53.643172 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.643153    2574 apiserver.go:52] "Watching apiserver"
Nov 04 01:54:53.664090 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.664069    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:54:53.668313 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.668299    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 04 01:54:53.742942 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.742902    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal" event={"ID":"f6b3c9ba0af4012b95306d2f2cfa3c61","Type":"ContainerStarted","Data":"c90dfcd721e4bf20cab1e57beb6127a78cb16c800014c20d17192be40af26ba5"}
Nov 04 01:54:53.744095 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.744072    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal" event={"ID":"56dfafb26cd80e2f2f640fddc6c0403a","Type":"ContainerStarted","Data":"a4cf7f99d75dbefad68a1a3ecfe1f4bc0aa617025acd7fe6bc3aa4552455a86a"}
Nov 04 01:54:53.755466 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.755448    2574 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 01:54:53.755748 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.755731    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-x48xl","kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal","openshift-cluster-node-tuning-operator/tuned-h4glm","openshift-dns/node-resolver-pj6wl","openshift-multus/multus-n5mdk","kube-system/konnectivity-agent-dszxl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64","openshift-multus/multus-additional-cni-plugins-78wz6","openshift-multus/network-metrics-daemon-74j82","openshift-network-diagnostics/network-check-target-g6xmn","openshift-ovn-kubernetes/ovnkube-node-k76ph","openshift-image-registry/node-ca-ntp6j","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal","openshift-network-operator/iptables-alerter-2s59r"]
Nov 04 01:54:53.759610 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.759596    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:54:53.759671 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:53.759656    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g6xmn" podUID="213808a8-b7ec-4fc6-845b-d2648393d5bc"
Nov 04 01:54:53.761755 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.761742    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:54:53.764015 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.763998    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.766269 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.766250    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:54:53.766352 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:53.766332    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x48xl" podUID="a994b38f-0096-4fff-b49c-e6d99dc5a6be"
Nov 04 01:54:53.770940 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.768864    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.771372 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.771351    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dszxl"
Nov 04 01:54:53.774060 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.774042    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.776602 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.776581    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.777221 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777202    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30efbdba-8299-418b-b8f1-233c789443f5-cni-binary-copy\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.777304 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777224    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff8251b4-1f23-4a86-9b1d-602c87cf133b-hosts-file\") pod \"node-resolver-pj6wl\" (UID: \"ff8251b4-1f23-4a86-9b1d-602c87cf133b\") " pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:54:53.777304 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777251    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-var-lib-kubelet\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.777304 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777275    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-host\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.777525 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777366    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-tuned\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.777525 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777418    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56z2g\" (UniqueName: \"kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g\") pod \"network-check-target-g6xmn\" (UID: \"213808a8-b7ec-4fc6-845b-d2648393d5bc\") " pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:54:53.777525 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777456    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-modprobe-d\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.777525 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777482    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-k8s-cni-cncf-io\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.777525 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777507    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-cni-multus\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.777770 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777548    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-lib-modules\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.777770 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777578    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a994b38f-0096-4fff-b49c-e6d99dc5a6be-kubelet-config\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:54:53.777770 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777594    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxwj7\" (UniqueName: \"kubernetes.io/projected/30efbdba-8299-418b-b8f1-233c789443f5-kube-api-access-lxwj7\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.777770 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777636    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-885ds\" (UniqueName: \"kubernetes.io/projected/ff8251b4-1f23-4a86-9b1d-602c87cf133b-kube-api-access-885ds\") pod \"node-resolver-pj6wl\" (UID: \"ff8251b4-1f23-4a86-9b1d-602c87cf133b\") " pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:54:53.777770 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777677    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a994b38f-0096-4fff-b49c-e6d99dc5a6be-dbus\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:54:53.777770 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777702    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff8251b4-1f23-4a86-9b1d-602c87cf133b-tmp-dir\") pod \"node-resolver-pj6wl\" (UID: \"ff8251b4-1f23-4a86-9b1d-602c87cf133b\") " pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:54:53.777770 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777739    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-kubernetes\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.777770 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777767    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysctl-conf\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.778188 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777802    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-systemd\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.778188 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777845    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-system-cni-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.778188 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777878    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-netns\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.778188 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777894    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-hostroot\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.778188 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777959    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-multus-certs\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.778188 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.777981    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-run\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.778188 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.778007    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-tmp\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.778188 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.778031    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-conf-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.778188 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.778046    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-cni-bin\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.778188 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.778098    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-cnibin\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.778188 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.778118    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-socket-dir-parent\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.778188 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.778136    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-kubelet\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.778188 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.778164    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysctl-d\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.778937 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.778219    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:54:53.778937 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.778236    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/30efbdba-8299-418b-b8f1-233c789443f5-multus-daemon-config\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.778937 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.778277    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-etc-kubernetes\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.778937 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.778316    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-sys\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.778937 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.778353    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-cni-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.778937 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.778371    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysconfig\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.778937 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.778387    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdkv4\" (UniqueName: \"kubernetes.io/projected/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-kube-api-access-gdkv4\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.778937 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.778936    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:54:53.779340 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:53.778987    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74j82" podUID="1536e272-39b6-4702-a35d-0130eef3e290"
Nov 04 01:54:53.781554 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.781537    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.784083 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.784061    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:54:53.786499 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.786481    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:54:53.819098 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.819074    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-s4jjj\""
Nov 04 01:54:53.819195 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.819135    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 01:54:53.819325 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.819310    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 01:54:53.820810 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.820790    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:54:53.821071 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.821055    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zhvgj\""
Nov 04 01:54:53.821123 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.821102    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 01:54:53.825917 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.825902    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 01:54:53.826062 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.826048    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 01:54:53.826200 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.826189    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-zk98g\""
Nov 04 01:54:53.826515 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.826498    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 01:54:53.826558 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.826515    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 01:54:53.828432 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.828400    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 01:54:53.828497 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.828481    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-tlvkz\""
Nov 04 01:54:53.828584 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.828573    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 01:54:53.831017 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.831002    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7qkcb\""
Nov 04 01:54:53.831227 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.831213    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 01:54:53.831281 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.831270    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 01:54:53.831319 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.831280    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 01:54:53.833351 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.833337    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 01:54:53.833683 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.833669    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bxxxw\""
Nov 04 01:54:53.833885 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.833873    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 01:54:53.838632 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.838617    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 01:54:53.838678 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.838660    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r7nnn\""
Nov 04 01:54:53.838900 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.838881    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 01:54:53.838970 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.838959    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 01:54:53.841241 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.839433    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 01:54:53.867914 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.867894    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:54:53.868020 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.867978    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 01:54:53.868020 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.867978    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nrwjt\""
Nov 04 01:54:53.868102 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.868027    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 01:54:53.868102 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.868040    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 01:54:53.868102 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.868052    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 01:54:53.868287 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.868271    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 01:54:53.868367 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.868348    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 01:54:53.868367 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.868355    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bzrbt\""
Nov 04 01:54:53.868442 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.868384    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 01:54:53.868442 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.868423    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 01:54:53.870581 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.870564    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 01:54:53.870625 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.870615    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cb2d9"
Nov 04 01:54:53.879097 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879080    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovnkube-script-lib\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.879148 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879103    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8lmb\" (UniqueName: \"kubernetes.io/projected/b2a74b29-e9ea-4ab9-b155-795d08684fe6-kube-api-access-c8lmb\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.879148 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879123    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tjn7\" (UniqueName: \"kubernetes.io/projected/c73834fc-41c8-4eea-a739-52d16535890d-kube-api-access-9tjn7\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.879148 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879142    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30efbdba-8299-418b-b8f1-233c789443f5-cni-binary-copy\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.879236 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879157    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-var-lib-kubelet\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.879236 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879206    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-host\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.879314 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879264    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-tuned\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.879314 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879285    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-cni-binary-copy\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.879393 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879319    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.879393 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879338    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56z2g\" (UniqueName: \"kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g\") pod \"network-check-target-g6xmn\" (UID: \"213808a8-b7ec-4fc6-845b-d2648393d5bc\") " pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:54:53.879393 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879364    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-lib-modules\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.879393 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879375    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-var-lib-kubelet\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.879393 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879390    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-node-log\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.879616 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879392    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-host\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.879616 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879439    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a994b38f-0096-4fff-b49c-e6d99dc5a6be-kubelet-config\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:54:53.879616 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879503    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-885ds\" (UniqueName: \"kubernetes.io/projected/ff8251b4-1f23-4a86-9b1d-602c87cf133b-kube-api-access-885ds\") pod \"node-resolver-pj6wl\" (UID: \"ff8251b4-1f23-4a86-9b1d-602c87cf133b\") " pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:54:53.879616 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879520    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a994b38f-0096-4fff-b49c-e6d99dc5a6be-kubelet-config\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:54:53.879813 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879611    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-cnibin\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.879813 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879596    2574 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 01:54:53.879813 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879665    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-lib-modules\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.879813 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879739    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.879813 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879779    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30efbdba-8299-418b-b8f1-233c789443f5-cni-binary-copy\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.879813 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879808    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8trls\" (UniqueName: \"kubernetes.io/projected/1536e272-39b6-4702-a35d-0130eef3e290-kube-api-access-8trls\") pod \"network-metrics-daemon-74j82\" (UID: \"1536e272-39b6-4702-a35d-0130eef3e290\") " pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:54:53.880117 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879848    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28wg4\" (UniqueName: \"kubernetes.io/projected/1d895e08-f347-4b7c-b63e-26b06a3f6c10-kube-api-access-28wg4\") pod \"node-ca-ntp6j\" (UID: \"1d895e08-f347-4b7c-b63e-26b06a3f6c10\") " pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:54:53.880117 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879902    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a994b38f-0096-4fff-b49c-e6d99dc5a6be-dbus\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:54:53.880117 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879933    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff8251b4-1f23-4a86-9b1d-602c87cf133b-tmp-dir\") pod \"node-resolver-pj6wl\" (UID: \"ff8251b4-1f23-4a86-9b1d-602c87cf133b\") " pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:54:53.880117 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879960    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-kubernetes\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.880117 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.879985    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysctl-conf\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.880117 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880012    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-systemd\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.880117 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880028    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-kubernetes\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.880117 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880041    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.880117 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880062    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a994b38f-0096-4fff-b49c-e6d99dc5a6be-dbus\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:54:53.880117 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880096    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbn5k\" (UniqueName: \"kubernetes.io/projected/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-kube-api-access-nbn5k\") pod \"iptables-alerter-2s59r\" (UID: \"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13\") " pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880120    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-systemd\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880124    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-system-cni-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880153    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-netns\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880173    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysctl-conf\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880179    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-multus-certs\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880206    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff8251b4-1f23-4a86-9b1d-602c87cf133b-tmp-dir\") pod \"node-resolver-pj6wl\" (UID: \"ff8251b4-1f23-4a86-9b1d-602c87cf133b\") " pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880221    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-netns\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880205    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-run\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880244    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-multus-certs\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880257    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-etc-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880276    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-device-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880256    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-run\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880202    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-system-cni-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880319    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-cni-bin\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880355    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-cnibin\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880391    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-kubelet\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880409    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-cni-bin\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.880549 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880429    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-cnibin\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.881288 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880456    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-cni-netd\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.881288 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880476    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1d895e08-f347-4b7c-b63e-26b06a3f6c10-serviceca\") pod \"node-ca-ntp6j\" (UID: \"1d895e08-f347-4b7c-b63e-26b06a3f6c10\") " pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:54:53.881288 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880533    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:54:53.881288 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880537    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-kubelet\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.881288 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880568    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/30efbdba-8299-418b-b8f1-233c789443f5-multus-daemon-config\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.881288 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880605    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1926369-9621-4cac-b797-43a306bdb709-konnectivity-ca\") pod \"konnectivity-agent-dszxl\" (UID: \"b1926369-9621-4cac-b797-43a306bdb709\") " pod="kube-system/konnectivity-agent-dszxl"
Nov 04 01:54:53.881288 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880623    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-cni-bin\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.881288 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:53.880630    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:54:53.881288 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880682    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-socket-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.881288 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:53.880716    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret podName:a994b38f-0096-4fff-b49c-e6d99dc5a6be nodeName:}" failed. No retries permitted until 2026-11-04 01:54:54.380700657 +0000 UTC m=+3.102736118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret") pod "global-pull-secret-syncer-x48xl" (UID: "a994b38f-0096-4fff-b49c-e6d99dc5a6be") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:54:53.881288 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880732    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysconfig\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.881288 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880760    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-var-lib-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.881288 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880814    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysconfig\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.881288 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880831    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.881288 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880868    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovnkube-config\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.881288 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880902    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-env-overrides\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.881288 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880927    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d895e08-f347-4b7c-b63e-26b06a3f6c10-host\") pod \"node-ca-ntp6j\" (UID: \"1d895e08-f347-4b7c-b63e-26b06a3f6c10\") " pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:54:53.882059 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.880973    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff8251b4-1f23-4a86-9b1d-602c87cf133b-hosts-file\") pod \"node-resolver-pj6wl\" (UID: \"ff8251b4-1f23-4a86-9b1d-602c87cf133b\") " pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:54:53.882059 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881004    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-systemd-units\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.882059 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881021    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-run-ovn-kubernetes\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.882059 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881070    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.882059 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881078    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff8251b4-1f23-4a86-9b1d-602c87cf133b-hosts-file\") pod \"node-resolver-pj6wl\" (UID: \"ff8251b4-1f23-4a86-9b1d-602c87cf133b\") " pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:54:53.882059 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881093    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/30efbdba-8299-418b-b8f1-233c789443f5-multus-daemon-config\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.882059 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881104    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-modprobe-d\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.882059 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881168    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-k8s-cni-cncf-io\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.882059 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881180    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-modprobe-d\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.882059 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881218    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-cni-multus\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.882059 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881240    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-k8s-cni-cncf-io\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.882059 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881265    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-slash\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.882059 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881280    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-systemd\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.882059 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881328    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-sys-fs\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.882059 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881348    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-cni-multus\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.882059 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881365    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxwj7\" (UniqueName: \"kubernetes.io/projected/30efbdba-8299-418b-b8f1-233c789443f5-kube-api-access-lxwj7\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.882059 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881397    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-ovn\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.882691 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881425    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-host-slash\") pod \"iptables-alerter-2s59r\" (UID: \"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13\") " pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:54:53.882691 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881457    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-registration-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.882691 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881482    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-etc-selinux\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.882691 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881510    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-system-cni-dir\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.882691 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881536    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs\") pod \"network-metrics-daemon-74j82\" (UID: \"1536e272-39b6-4702-a35d-0130eef3e290\") " pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:54:53.882691 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881554    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-hostroot\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.882691 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881610    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-tmp\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.882691 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881618    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-hostroot\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.882691 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881648    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-log-socket\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.882691 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881694    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovn-node-metrics-cert\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.882691 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881740    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-iptables-alerter-script\") pod \"iptables-alerter-2s59r\" (UID: \"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13\") " pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:54:53.882691 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881770    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sfr5\" (UniqueName: \"kubernetes.io/projected/0aa5b60a-ce95-4623-8ae2-32896ca1232d-kube-api-access-4sfr5\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.882691 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881825    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-conf-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.882691 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881857    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-socket-dir-parent\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.882691 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881896    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-conf-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.882691 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881915    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysctl-d\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.882691 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881929    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-socket-dir-parent\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.883220 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.881956    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-run-netns\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.883220 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.882002    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-etc-kubernetes\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.883220 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.882020    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-sys\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.883220 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.882030    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysctl-d\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.883220 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.882060    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-kubelet\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.883220 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.882062    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-etc-kubernetes\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.883220 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.882094    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.883220 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.882117    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-sys\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.883220 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.882126    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-cni-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.883220 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.882169    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdkv4\" (UniqueName: \"kubernetes.io/projected/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-kube-api-access-gdkv4\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.883220 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.882225    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-cni-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.883220 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.882249    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1926369-9621-4cac-b797-43a306bdb709-agent-certs\") pod \"konnectivity-agent-dszxl\" (UID: \"b1926369-9621-4cac-b797-43a306bdb709\") " pod="kube-system/konnectivity-agent-dszxl"
Nov 04 01:54:53.883220 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.883019    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-tuned\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.884284 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.884267    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-tmp\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.943673 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:53.943654    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:54:53.943673 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:53.943675    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:54:53.943795 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:53.943685    2574 projected.go:196] Error preparing data for projected volume kube-api-access-56z2g for pod openshift-network-diagnostics/network-check-target-g6xmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:54:53.943795 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:53.943746    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g podName:213808a8-b7ec-4fc6-845b-d2648393d5bc nodeName:}" failed. No retries permitted until 2026-11-04 01:54:54.443727852 +0000 UTC m=+3.165763298 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-56z2g" (UniqueName: "kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g") pod "network-check-target-g6xmn" (UID: "213808a8-b7ec-4fc6-845b-d2648393d5bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:54:53.945875 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.945832    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-885ds\" (UniqueName: \"kubernetes.io/projected/ff8251b4-1f23-4a86-9b1d-602c87cf133b-kube-api-access-885ds\") pod \"node-resolver-pj6wl\" (UID: \"ff8251b4-1f23-4a86-9b1d-602c87cf133b\") " pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:54:53.946795 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.946778    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxwj7\" (UniqueName: \"kubernetes.io/projected/30efbdba-8299-418b-b8f1-233c789443f5-kube-api-access-lxwj7\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:53.947256 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.947235    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdkv4\" (UniqueName: \"kubernetes.io/projected/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-kube-api-access-gdkv4\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:53.982490 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982472    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1926369-9621-4cac-b797-43a306bdb709-konnectivity-ca\") pod \"konnectivity-agent-dszxl\" (UID: \"b1926369-9621-4cac-b797-43a306bdb709\") " pod="kube-system/konnectivity-agent-dszxl"
Nov 04 01:54:53.982551 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982497    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-cni-bin\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.982551 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982512    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-socket-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.982551 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982529    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-var-lib-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.982551 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982545    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.982750 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982567    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovnkube-config\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.982750 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982573    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-cni-bin\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.982750 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982593    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-env-overrides\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.982750 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982624    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d895e08-f347-4b7c-b63e-26b06a3f6c10-host\") pod \"node-ca-ntp6j\" (UID: \"1d895e08-f347-4b7c-b63e-26b06a3f6c10\") " pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:54:53.982750 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982647    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-systemd-units\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.982750 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982637    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.982750 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982678    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-run-ovn-kubernetes\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.982750 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982693    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-socket-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.982750 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982706    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.982750 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982717    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d895e08-f347-4b7c-b63e-26b06a3f6c10-host\") pod \"node-ca-ntp6j\" (UID: \"1d895e08-f347-4b7c-b63e-26b06a3f6c10\") " pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:54:53.982750 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982734    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-slash\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.982750 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982719    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-systemd-units\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.982750 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982743    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-run-ovn-kubernetes\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.983345 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982761    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-systemd\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.983345 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982786    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-slash\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.983345 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982785    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-sys-fs\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.983345 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982794    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.983345 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982820    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-ovn\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.983345 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982831    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-sys-fs\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.983345 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982840    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-systemd\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.983345 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982848    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-host-slash\") pod \"iptables-alerter-2s59r\" (UID: \"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13\") " pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:54:53.983345 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982877    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-ovn\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.983345 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982881    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-registration-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.983345 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982900    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-host-slash\") pod \"iptables-alerter-2s59r\" (UID: \"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13\") " pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:54:53.983345 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982907    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-etc-selinux\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.983345 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982916    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-var-lib-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.983345 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982957    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-registration-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.983345 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982982    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-system-cni-dir\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.983345 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983010    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs\") pod \"network-metrics-daemon-74j82\" (UID: \"1536e272-39b6-4702-a35d-0130eef3e290\") " pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:54:53.983345 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983016    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-system-cni-dir\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.984251 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.982984    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-etc-selinux\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.984251 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983037    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-log-socket\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.984251 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983072    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovn-node-metrics-cert\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.984251 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983081    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-log-socket\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.984251 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:53.983100    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:54:53.984251 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983108    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-env-overrides\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.984251 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983108    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1926369-9621-4cac-b797-43a306bdb709-konnectivity-ca\") pod \"konnectivity-agent-dszxl\" (UID: \"b1926369-9621-4cac-b797-43a306bdb709\") " pod="kube-system/konnectivity-agent-dszxl"
Nov 04 01:54:53.984251 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983098    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-iptables-alerter-script\") pod \"iptables-alerter-2s59r\" (UID: \"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13\") " pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:54:53.984251 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:53.983146    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs podName:1536e272-39b6-4702-a35d-0130eef3e290 nodeName:}" failed. No retries permitted until 2026-11-04 01:54:54.48313156 +0000 UTC m=+3.205167020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs") pod "network-metrics-daemon-74j82" (UID: "1536e272-39b6-4702-a35d-0130eef3e290") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:54:53.984251 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983168    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovnkube-config\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.984251 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983181    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sfr5\" (UniqueName: \"kubernetes.io/projected/0aa5b60a-ce95-4623-8ae2-32896ca1232d-kube-api-access-4sfr5\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.984251 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983220    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-run-netns\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.984251 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983250    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-kubelet\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.984251 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983254    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-run-netns\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.984251 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983278    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.984251 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983309    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1926369-9621-4cac-b797-43a306bdb709-agent-certs\") pod \"konnectivity-agent-dszxl\" (UID: \"b1926369-9621-4cac-b797-43a306bdb709\") " pod="kube-system/konnectivity-agent-dszxl"
Nov 04 01:54:53.984251 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983314    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-kubelet\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.985211 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983334    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovnkube-script-lib\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.985211 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983380    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8lmb\" (UniqueName: \"kubernetes.io/projected/b2a74b29-e9ea-4ab9-b155-795d08684fe6-kube-api-access-c8lmb\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.985211 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983407    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tjn7\" (UniqueName: \"kubernetes.io/projected/c73834fc-41c8-4eea-a739-52d16535890d-kube-api-access-9tjn7\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.985211 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983439    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-cni-binary-copy\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.985211 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983464    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.985211 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983500    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-node-log\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.985211 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983527    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-cnibin\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.985211 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983551    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.985211 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983576    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8trls\" (UniqueName: \"kubernetes.io/projected/1536e272-39b6-4702-a35d-0130eef3e290-kube-api-access-8trls\") pod \"network-metrics-daemon-74j82\" (UID: \"1536e272-39b6-4702-a35d-0130eef3e290\") " pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:54:53.985211 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983608    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28wg4\" (UniqueName: \"kubernetes.io/projected/1d895e08-f347-4b7c-b63e-26b06a3f6c10-kube-api-access-28wg4\") pod \"node-ca-ntp6j\" (UID: \"1d895e08-f347-4b7c-b63e-26b06a3f6c10\") " pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:54:53.985211 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983637    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-iptables-alerter-script\") pod \"iptables-alerter-2s59r\" (UID: \"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13\") " pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:54:53.985211 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983673    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.985211 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983728    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.985211 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983821    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.985211 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983874    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.985211 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983883    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-cnibin\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.985760 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983874    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbn5k\" (UniqueName: \"kubernetes.io/projected/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-kube-api-access-nbn5k\") pod \"iptables-alerter-2s59r\" (UID: \"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13\") " pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:54:53.985760 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983959    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-etc-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.985760 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.983989    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-device-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.985760 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.984019    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-cni-netd\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.985760 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.984024    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-etc-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.985760 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.984045    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1d895e08-f347-4b7c-b63e-26b06a3f6c10-serviceca\") pod \"node-ca-ntp6j\" (UID: \"1d895e08-f347-4b7c-b63e-26b06a3f6c10\") " pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:54:53.985760 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.984052    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-node-log\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.985760 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.984080    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-device-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:53.985760 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.984110    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-cni-netd\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.985760 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.984155    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.985760 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.984309    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovnkube-script-lib\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.985760 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.984409    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-cni-binary-copy\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:53.985760 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.984538    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1d895e08-f347-4b7c-b63e-26b06a3f6c10-serviceca\") pod \"node-ca-ntp6j\" (UID: \"1d895e08-f347-4b7c-b63e-26b06a3f6c10\") " pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:54:53.985760 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.985513    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovn-node-metrics-cert\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:53.986172 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:53.985881    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1926369-9621-4cac-b797-43a306bdb709-agent-certs\") pod \"konnectivity-agent-dszxl\" (UID: \"b1926369-9621-4cac-b797-43a306bdb709\") " pod="kube-system/konnectivity-agent-dszxl"
Nov 04 01:54:54.050498 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.050474    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbn5k\" (UniqueName: \"kubernetes.io/projected/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-kube-api-access-nbn5k\") pod \"iptables-alerter-2s59r\" (UID: \"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13\") " pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:54:54.050498 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.050485    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sfr5\" (UniqueName: \"kubernetes.io/projected/0aa5b60a-ce95-4623-8ae2-32896ca1232d-kube-api-access-4sfr5\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:54.050922 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.050899    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tjn7\" (UniqueName: \"kubernetes.io/projected/c73834fc-41c8-4eea-a739-52d16535890d-kube-api-access-9tjn7\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:54.051012 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.050968    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28wg4\" (UniqueName: \"kubernetes.io/projected/1d895e08-f347-4b7c-b63e-26b06a3f6c10-kube-api-access-28wg4\") pod \"node-ca-ntp6j\" (UID: \"1d895e08-f347-4b7c-b63e-26b06a3f6c10\") " pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:54:54.051012 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.051000    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8trls\" (UniqueName: \"kubernetes.io/projected/1536e272-39b6-4702-a35d-0130eef3e290-kube-api-access-8trls\") pod \"network-metrics-daemon-74j82\" (UID: \"1536e272-39b6-4702-a35d-0130eef3e290\") " pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:54:54.051177 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.051162    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8lmb\" (UniqueName: \"kubernetes.io/projected/b2a74b29-e9ea-4ab9-b155-795d08684fe6-kube-api-access-c8lmb\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:54.074067 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.074021    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:54:54.080511 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.080488    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:54:54.084741 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:54.084721    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff8251b4_1f23_4a86_9b1d_602c87cf133b.slice/crio-7f966c4979ac588dd8a85320db55bac8c3612d048637276dc06f572e05654ea7 WatchSource:0}: Error finding container 7f966c4979ac588dd8a85320db55bac8c3612d048637276dc06f572e05654ea7: Status 404 returned error can't find the container with id 7f966c4979ac588dd8a85320db55bac8c3612d048637276dc06f572e05654ea7
Nov 04 01:54:54.089322 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:54.089301    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2522cd43_48a3_40d4_9f8f_b7a0f778abbd.slice/crio-1245fd407f118f76ac0f93fa2b6dca16c4b9477ae95ed4569c5cc9e24c3c55dd WatchSource:0}: Error finding container 1245fd407f118f76ac0f93fa2b6dca16c4b9477ae95ed4569c5cc9e24c3c55dd: Status 404 returned error can't find the container with id 1245fd407f118f76ac0f93fa2b6dca16c4b9477ae95ed4569c5cc9e24c3c55dd
Nov 04 01:54:54.089904 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.089887    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n5mdk"
Nov 04 01:54:54.092572 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.092553    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dszxl"
Nov 04 01:54:54.100051 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:54.100021    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30efbdba_8299_418b_b8f1_233c789443f5.slice/crio-629c45f7398cd6c8a2ecb52ed62614f33e8715c09e20a7bceb39ec7706ee998d WatchSource:0}: Error finding container 629c45f7398cd6c8a2ecb52ed62614f33e8715c09e20a7bceb39ec7706ee998d: Status 404 returned error can't find the container with id 629c45f7398cd6c8a2ecb52ed62614f33e8715c09e20a7bceb39ec7706ee998d
Nov 04 01:54:54.103508 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.103488    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:54:54.103726 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:54.103708    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1926369_9621_4cac_b797_43a306bdb709.slice/crio-b101aa382934986eaf277925805516432fd4e9bc320282679775888f7a46b1a1 WatchSource:0}: Error finding container b101aa382934986eaf277925805516432fd4e9bc320282679775888f7a46b1a1: Status 404 returned error can't find the container with id b101aa382934986eaf277925805516432fd4e9bc320282679775888f7a46b1a1
Nov 04 01:54:54.112603 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:54.112584    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa5b60a_ce95_4623_8ae2_32896ca1232d.slice/crio-8affe663e5e234923e78515c2f437f227257c62324a262f64ddbab5ca6a92bff WatchSource:0}: Error finding container 8affe663e5e234923e78515c2f437f227257c62324a262f64ddbab5ca6a92bff: Status 404 returned error can't find the container with id 8affe663e5e234923e78515c2f437f227257c62324a262f64ddbab5ca6a92bff
Nov 04 01:54:54.123040 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.123020    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:54:54.126468 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.126452    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:54:54.128845 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.128828    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:54:54.132908 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:54.132885    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d895e08_f347_4b7c_b63e_26b06a3f6c10.slice/crio-4f4681af6a37a15af43b80bc1cde4a59db4649601f60f1bc9e4323af8e8b2b81 WatchSource:0}: Error finding container 4f4681af6a37a15af43b80bc1cde4a59db4649601f60f1bc9e4323af8e8b2b81: Status 404 returned error can't find the container with id 4f4681af6a37a15af43b80bc1cde4a59db4649601f60f1bc9e4323af8e8b2b81
Nov 04 01:54:54.136437 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:54.136413    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73834fc_41c8_4eea_a739_52d16535890d.slice/crio-e203d9f7f4dace84a1605cb1848bc64e2806a328e3bf76639b08b554fc584054 WatchSource:0}: Error finding container e203d9f7f4dace84a1605cb1848bc64e2806a328e3bf76639b08b554fc584054: Status 404 returned error can't find the container with id e203d9f7f4dace84a1605cb1848bc64e2806a328e3bf76639b08b554fc584054
Nov 04 01:54:54.138709 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:54.138686    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f2a882_c7a7_4227_9c3e_bbfb5fbadf13.slice/crio-9cb410ae7eb6035bbc16b4def3d7faf5c3582513da295e519a4751b2a7d761f6 WatchSource:0}: Error finding container 9cb410ae7eb6035bbc16b4def3d7faf5c3582513da295e519a4751b2a7d761f6: Status 404 returned error can't find the container with id 9cb410ae7eb6035bbc16b4def3d7faf5c3582513da295e519a4751b2a7d761f6
Nov 04 01:54:54.139633 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.139612    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:54:54.148896 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:54:54.148878    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2a74b29_e9ea_4ab9_b155_795d08684fe6.slice/crio-37b91df8e28645c78cae6da6ead79ba0763b96b9e62d36b79cae5566e8a3ab50 WatchSource:0}: Error finding container 37b91df8e28645c78cae6da6ead79ba0763b96b9e62d36b79cae5566e8a3ab50: Status 404 returned error can't find the container with id 37b91df8e28645c78cae6da6ead79ba0763b96b9e62d36b79cae5566e8a3ab50
Nov 04 01:54:54.386724 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.386641    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:54:54.386866 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:54.386767    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:54:54.386866 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:54.386818    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret podName:a994b38f-0096-4fff-b49c-e6d99dc5a6be nodeName:}" failed. No retries permitted until 2026-11-04 01:54:55.386805155 +0000 UTC m=+4.108840596 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret") pod "global-pull-secret-syncer-x48xl" (UID: "a994b38f-0096-4fff-b49c-e6d99dc5a6be") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:54:54.486953 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.486922    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs\") pod \"network-metrics-daemon-74j82\" (UID: \"1536e272-39b6-4702-a35d-0130eef3e290\") " pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:54:54.487129 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.486982    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56z2g\" (UniqueName: \"kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g\") pod \"network-check-target-g6xmn\" (UID: \"213808a8-b7ec-4fc6-845b-d2648393d5bc\") " pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:54:54.487129 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:54.487070    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:54:54.487129 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:54.487114    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:54:54.487302 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:54.487134    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:54:54.487302 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:54.487144    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs podName:1536e272-39b6-4702-a35d-0130eef3e290 nodeName:}" failed. No retries permitted until 2026-11-04 01:54:55.487125238 +0000 UTC m=+4.209160682 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs") pod "network-metrics-daemon-74j82" (UID: "1536e272-39b6-4702-a35d-0130eef3e290") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:54:54.487302 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:54.487150    2574 projected.go:196] Error preparing data for projected volume kube-api-access-56z2g for pod openshift-network-diagnostics/network-check-target-g6xmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:54:54.487302 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:54.487187    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g podName:213808a8-b7ec-4fc6-845b-d2648393d5bc nodeName:}" failed. No retries permitted until 2026-11-04 01:54:55.487177359 +0000 UTC m=+4.209212800 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-56z2g" (UniqueName: "kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g") pod "network-check-target-g6xmn" (UID: "213808a8-b7ec-4fc6-845b-d2648393d5bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:54:54.748107 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.748002    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-78wz6" event={"ID":"c73834fc-41c8-4eea-a739-52d16535890d","Type":"ContainerStarted","Data":"e203d9f7f4dace84a1605cb1848bc64e2806a328e3bf76639b08b554fc584054"}
Nov 04 01:54:54.749449 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.749417    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dszxl" event={"ID":"b1926369-9621-4cac-b797-43a306bdb709","Type":"ContainerStarted","Data":"b101aa382934986eaf277925805516432fd4e9bc320282679775888f7a46b1a1"}
Nov 04 01:54:54.750724 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.750703    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n5mdk" event={"ID":"30efbdba-8299-418b-b8f1-233c789443f5","Type":"ContainerStarted","Data":"629c45f7398cd6c8a2ecb52ed62614f33e8715c09e20a7bceb39ec7706ee998d"}
Nov 04 01:54:54.752097 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.752071    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pj6wl" event={"ID":"ff8251b4-1f23-4a86-9b1d-602c87cf133b","Type":"ContainerStarted","Data":"7f966c4979ac588dd8a85320db55bac8c3612d048637276dc06f572e05654ea7"}
Nov 04 01:54:54.753406 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.753378    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph" event={"ID":"b2a74b29-e9ea-4ab9-b155-795d08684fe6","Type":"ContainerStarted","Data":"37b91df8e28645c78cae6da6ead79ba0763b96b9e62d36b79cae5566e8a3ab50"}
Nov 04 01:54:54.754588 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.754562    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ntp6j" event={"ID":"1d895e08-f347-4b7c-b63e-26b06a3f6c10","Type":"ContainerStarted","Data":"4f4681af6a37a15af43b80bc1cde4a59db4649601f60f1bc9e4323af8e8b2b81"}
Nov 04 01:54:54.756024 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.755969    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64" event={"ID":"0aa5b60a-ce95-4623-8ae2-32896ca1232d","Type":"ContainerStarted","Data":"8affe663e5e234923e78515c2f437f227257c62324a262f64ddbab5ca6a92bff"}
Nov 04 01:54:54.757275 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.757250    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h4glm" event={"ID":"2522cd43-48a3-40d4-9f8f-b7a0f778abbd","Type":"ContainerStarted","Data":"1245fd407f118f76ac0f93fa2b6dca16c4b9477ae95ed4569c5cc9e24c3c55dd"}
Nov 04 01:54:54.758773 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.758750    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2s59r" event={"ID":"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13","Type":"ContainerStarted","Data":"9cb410ae7eb6035bbc16b4def3d7faf5c3582513da295e519a4751b2a7d761f6"}
Nov 04 01:54:54.873765 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.873689    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 01:49:54 +0000 UTC" deadline="2028-05-03 06:40:53.912615373 +0000 UTC"
Nov 04 01:54:54.873765 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:54.873757    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13108h45m59.038861485s"
Nov 04 01:54:55.394513 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:55.394472    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:54:55.394715 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:55.394646    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:54:55.394715 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:55.394709    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret podName:a994b38f-0096-4fff-b49c-e6d99dc5a6be nodeName:}" failed. No retries permitted until 2026-11-04 01:54:57.394691659 +0000 UTC m=+6.116727103 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret") pod "global-pull-secret-syncer-x48xl" (UID: "a994b38f-0096-4fff-b49c-e6d99dc5a6be") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:54:55.495444 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:55.495406    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56z2g\" (UniqueName: \"kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g\") pod \"network-check-target-g6xmn\" (UID: \"213808a8-b7ec-4fc6-845b-d2648393d5bc\") " pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:54:55.495634 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:55.495499    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs\") pod \"network-metrics-daemon-74j82\" (UID: \"1536e272-39b6-4702-a35d-0130eef3e290\") " pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:54:55.495634 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:55.495620    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:54:55.495774 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:55.495679    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs podName:1536e272-39b6-4702-a35d-0130eef3e290 nodeName:}" failed. No retries permitted until 2026-11-04 01:54:57.495660856 +0000 UTC m=+6.217696300 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs") pod "network-metrics-daemon-74j82" (UID: "1536e272-39b6-4702-a35d-0130eef3e290") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:54:55.496066 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:55.496045    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:54:55.496135 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:55.496072    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:54:55.496135 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:55.496086    2574 projected.go:196] Error preparing data for projected volume kube-api-access-56z2g for pod openshift-network-diagnostics/network-check-target-g6xmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:54:55.496135 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:55.496129    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g podName:213808a8-b7ec-4fc6-845b-d2648393d5bc nodeName:}" failed. No retries permitted until 2026-11-04 01:54:57.496115317 +0000 UTC m=+6.218150773 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-56z2g" (UniqueName: "kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g") pod "network-check-target-g6xmn" (UID: "213808a8-b7ec-4fc6-845b-d2648393d5bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:54:55.745844 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:55.745772    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:54:55.746005 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:55.745915    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x48xl" podUID="a994b38f-0096-4fff-b49c-e6d99dc5a6be"
Nov 04 01:54:55.746299 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:55.746266    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:54:55.746439 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:55.746387    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g6xmn" podUID="213808a8-b7ec-4fc6-845b-d2648393d5bc"
Nov 04 01:54:55.746508 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:55.746451    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:54:55.746570 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:55.746540    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74j82" podUID="1536e272-39b6-4702-a35d-0130eef3e290"
Nov 04 01:54:55.874629 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:55.874562    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 01:49:54 +0000 UTC" deadline="2028-05-05 16:16:14.032130548 +0000 UTC"
Nov 04 01:54:55.874629 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:55.874627    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13166h21m18.157507731s"
Nov 04 01:54:57.411813 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:57.411551    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:54:57.412408 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:57.411958    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:54:57.412408 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:57.412025    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret podName:a994b38f-0096-4fff-b49c-e6d99dc5a6be nodeName:}" failed. No retries permitted until 2026-11-04 01:55:01.412005978 +0000 UTC m=+10.134041433 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret") pod "global-pull-secret-syncer-x48xl" (UID: "a994b38f-0096-4fff-b49c-e6d99dc5a6be") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:54:57.513438 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:57.513408    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs\") pod \"network-metrics-daemon-74j82\" (UID: \"1536e272-39b6-4702-a35d-0130eef3e290\") " pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:54:57.513556 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:57.513478    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56z2g\" (UniqueName: \"kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g\") pod \"network-check-target-g6xmn\" (UID: \"213808a8-b7ec-4fc6-845b-d2648393d5bc\") " pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:54:57.513681 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:57.513663    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:54:57.513753 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:57.513686    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:54:57.513753 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:57.513699    2574 projected.go:196] Error preparing data for projected volume kube-api-access-56z2g for pod openshift-network-diagnostics/network-check-target-g6xmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:54:57.513901 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:57.513754    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g podName:213808a8-b7ec-4fc6-845b-d2648393d5bc nodeName:}" failed. No retries permitted until 2026-11-04 01:55:01.513737014 +0000 UTC m=+10.235772459 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-56z2g" (UniqueName: "kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g") pod "network-check-target-g6xmn" (UID: "213808a8-b7ec-4fc6-845b-d2648393d5bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:54:57.513901 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:57.513826    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:54:57.513901 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:57.513865    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs podName:1536e272-39b6-4702-a35d-0130eef3e290 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:01.513852189 +0000 UTC m=+10.235887633 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs") pod "network-metrics-daemon-74j82" (UID: "1536e272-39b6-4702-a35d-0130eef3e290") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:54:57.740649 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:57.740477    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:54:57.740649 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:57.740622    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g6xmn" podUID="213808a8-b7ec-4fc6-845b-d2648393d5bc"
Nov 04 01:54:57.740826 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:57.740704    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:54:57.740826 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:57.740794    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74j82" podUID="1536e272-39b6-4702-a35d-0130eef3e290"
Nov 04 01:54:57.740909 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:57.740843    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:54:57.740950 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:57.740913    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x48xl" podUID="a994b38f-0096-4fff-b49c-e6d99dc5a6be"
Nov 04 01:54:57.794182 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:57.794142    2574 generic.go:356] "Generic (PLEG): container finished" podID="f6b3c9ba0af4012b95306d2f2cfa3c61" containerID="a3f943c27344712b7b62303a8f6610281c8e8dcb8fa74233e6f62d57b55f567e" exitCode=0
Nov 04 01:54:57.794570 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:57.794413    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal" event={"ID":"f6b3c9ba0af4012b95306d2f2cfa3c61","Type":"ContainerDied","Data":"a3f943c27344712b7b62303a8f6610281c8e8dcb8fa74233e6f62d57b55f567e"}
Nov 04 01:54:57.803958 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:57.803923    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal" event={"ID":"56dfafb26cd80e2f2f640fddc6c0403a","Type":"ContainerStarted","Data":"5d7ccd269c59882bdea6d0caf7e4253bfc46abcb8bccb64fa7ba6046468ea164"}
Nov 04 01:54:57.811264 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:57.811229    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-78wz6" event={"ID":"c73834fc-41c8-4eea-a739-52d16535890d","Type":"ContainerStarted","Data":"ca37d879af8640685efd83d07bcfc4c44d7d7482fc89c5f359d686e1d75d0115"}
Nov 04 01:54:58.825302 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:58.825249    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal" event={"ID":"f6b3c9ba0af4012b95306d2f2cfa3c61","Type":"ContainerStarted","Data":"adaed33514693cd33b85c3aab17c6b59116cea6c71318536f65b4acc8c5e295f"}
Nov 04 01:54:58.830554 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:58.829508    2574 generic.go:356] "Generic (PLEG): container finished" podID="c73834fc-41c8-4eea-a739-52d16535890d" containerID="ca37d879af8640685efd83d07bcfc4c44d7d7482fc89c5f359d686e1d75d0115" exitCode=0
Nov 04 01:54:58.830554 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:58.830489    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-78wz6" event={"ID":"c73834fc-41c8-4eea-a739-52d16535890d","Type":"ContainerDied","Data":"ca37d879af8640685efd83d07bcfc4c44d7d7482fc89c5f359d686e1d75d0115"}
Nov 04 01:54:58.951869 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:58.951797    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal" podStartSLOduration=4.951775986 podStartE2EDuration="4.951775986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:54:58.193280783 +0000 UTC m=+6.915316248" watchObservedRunningTime="2026-11-04 01:54:58.951775986 +0000 UTC m=+7.673811451"
Nov 04 01:54:59.086006 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:59.085950    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal" podStartSLOduration=5.0859299 podStartE2EDuration="5.0859299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:54:58.952926346 +0000 UTC m=+7.674961812" watchObservedRunningTime="2026-11-04 01:54:59.0859299 +0000 UTC m=+7.807965367"
Nov 04 01:54:59.741439 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:59.741405    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:54:59.741671 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:59.741566    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g6xmn" podUID="213808a8-b7ec-4fc6-845b-d2648393d5bc"
Nov 04 01:54:59.742010 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:59.741979    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:54:59.742127 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:59.742074    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74j82" podUID="1536e272-39b6-4702-a35d-0130eef3e290"
Nov 04 01:54:59.742407 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:59.742371    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:54:59.742492 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:54:59.742462    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x48xl" podUID="a994b38f-0096-4fff-b49c-e6d99dc5a6be"
Nov 04 01:54:59.835234 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:54:59.835199    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dszxl" event={"ID":"b1926369-9621-4cac-b797-43a306bdb709","Type":"ContainerStarted","Data":"c1b8a50b2c2ab990993ab9cb01a63408cf074241bc82aa49700762a1cae4a9a3"}
Nov 04 01:55:01.449777 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:01.449734    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:01.450439 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:01.449936    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:55:01.450439 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:01.450005    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret podName:a994b38f-0096-4fff-b49c-e6d99dc5a6be nodeName:}" failed. No retries permitted until 2026-11-04 01:55:09.449986927 +0000 UTC m=+18.172022410 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret") pod "global-pull-secret-syncer-x48xl" (UID: "a994b38f-0096-4fff-b49c-e6d99dc5a6be") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:55:01.550709 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:01.550666    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs\") pod \"network-metrics-daemon-74j82\" (UID: \"1536e272-39b6-4702-a35d-0130eef3e290\") " pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:01.550888 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:01.550735    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56z2g\" (UniqueName: \"kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g\") pod \"network-check-target-g6xmn\" (UID: \"213808a8-b7ec-4fc6-845b-d2648393d5bc\") " pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:01.550962 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:01.550915    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:55:01.550962 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:01.550938    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:55:01.550962 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:01.550952    2574 projected.go:196] Error preparing data for projected volume kube-api-access-56z2g for pod openshift-network-diagnostics/network-check-target-g6xmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:55:01.551236 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:01.551013    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g podName:213808a8-b7ec-4fc6-845b-d2648393d5bc nodeName:}" failed. No retries permitted until 2026-11-04 01:55:09.550994125 +0000 UTC m=+18.273029580 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-56z2g" (UniqueName: "kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g") pod "network-check-target-g6xmn" (UID: "213808a8-b7ec-4fc6-845b-d2648393d5bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:55:01.551496 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:01.551458    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:55:01.551649 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:01.551516    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs podName:1536e272-39b6-4702-a35d-0130eef3e290 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:09.551499587 +0000 UTC m=+18.273535034 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs") pod "network-metrics-daemon-74j82" (UID: "1536e272-39b6-4702-a35d-0130eef3e290") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:55:01.740242 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:01.740155    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:01.740407 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:01.740311    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:01.740407 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:01.740340    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74j82" podUID="1536e272-39b6-4702-a35d-0130eef3e290"
Nov 04 01:55:01.740530 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:01.740443    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x48xl" podUID="a994b38f-0096-4fff-b49c-e6d99dc5a6be"
Nov 04 01:55:01.740530 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:01.740509    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:01.740640 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:01.740584    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g6xmn" podUID="213808a8-b7ec-4fc6-845b-d2648393d5bc"
Nov 04 01:55:01.842116 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:01.842070    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ntp6j" event={"ID":"1d895e08-f347-4b7c-b63e-26b06a3f6c10","Type":"ContainerStarted","Data":"73d4e46254e95074457275743c339acae3205244084a8080719c023b965c4e49"}
Nov 04 01:55:01.845280 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:01.845198    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64" event={"ID":"0aa5b60a-ce95-4623-8ae2-32896ca1232d","Type":"ContainerStarted","Data":"e38c4978e0f343d1391c0b50ab6d6016bb8782933b25438bb2bf9ed745fce307"}
Nov 04 01:55:01.968850 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:01.968785    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dszxl" podStartSLOduration=4.002434733 podStartE2EDuration="8.968765234s" totalImagesPullingTime="4.966330501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:54:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:54:59.961961893 +0000 UTC m=+8.683997359" watchObservedRunningTime="2026-11-04 01:55:01.968765234 +0000 UTC m=+10.690800702"
Nov 04 01:55:01.969447 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:01.969405    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ntp6j" podStartSLOduration=2.215434768 podStartE2EDuration="8.969394565s" totalImagesPullingTime="6.753959797s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:01.96829226 +0000 UTC m=+10.690327729" watchObservedRunningTime="2026-11-04 01:55:01.969394565 +0000 UTC m=+10.691430031"
Nov 04 01:55:03.418528 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:03.418382    2574 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 01:55:03.739448 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:03.739377    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:03.739608 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:03.739494    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g6xmn" podUID="213808a8-b7ec-4fc6-845b-d2648393d5bc"
Nov 04 01:55:03.739844 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:03.739821    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:03.739966 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:03.739937    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74j82" podUID="1536e272-39b6-4702-a35d-0130eef3e290"
Nov 04 01:55:03.740101 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:03.740016    2574 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T01:55:03.418404388Z","UUID":"ab1ee9bd-6bb9-4d1d-b82a-f6667c87ba8d","Handler":null,"Name":"","Endpoint":""}
Nov 04 01:55:03.740388 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:03.740365    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:03.740520 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:03.740489    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x48xl" podUID="a994b38f-0096-4fff-b49c-e6d99dc5a6be"
Nov 04 01:55:03.742426 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:03.742019    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 01:55:03.742426 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:03.742050    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 01:55:03.851915 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:03.851869    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n5mdk" event={"ID":"30efbdba-8299-418b-b8f1-233c789443f5","Type":"ContainerStarted","Data":"be1435a3b3603543c166ea20cdc90c59582b4f7bed9023e20b01d4933202e1ba"}
Nov 04 01:55:03.853521 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:03.853487    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pj6wl" event={"ID":"ff8251b4-1f23-4a86-9b1d-602c87cf133b","Type":"ContainerStarted","Data":"d00683b95601507fa956759773f8260bc7665e6e38eca3c133dc0cb6663bdfab"}
Nov 04 01:55:03.855847 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:03.855822    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64" event={"ID":"0aa5b60a-ce95-4623-8ae2-32896ca1232d","Type":"ContainerStarted","Data":"19c95d88f38bb978c3995b0cd60caafc097a6a69f35815cf385ae5c647e7a402"}
Nov 04 01:55:03.987922 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:03.987856    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-n5mdk" podStartSLOduration=2.261740475 podStartE2EDuration="10.987837247s" totalImagesPullingTime="8.726096772s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:03.987680436 +0000 UTC m=+12.709715901" watchObservedRunningTime="2026-11-04 01:55:03.987837247 +0000 UTC m=+12.709872713"
Nov 04 01:55:04.114402 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:04.114325    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-pj6wl" podStartSLOduration=2.042647335 podStartE2EDuration="11.114312413s" totalImagesPullingTime="9.071665078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:04.113467431 +0000 UTC m=+12.835502900" watchObservedRunningTime="2026-11-04 01:55:04.114312413 +0000 UTC m=+12.836347877"
Nov 04 01:55:04.466695 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:04.466666    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dszxl"
Nov 04 01:55:04.467257 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:04.467233    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dszxl"
Nov 04 01:55:04.860954 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:04.860863    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2s59r" event={"ID":"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13","Type":"ContainerStarted","Data":"37e5fde9b5ec226d0ca36c90f77310435ddb8478ac96764f865ae125a004284d"}
Nov 04 01:55:04.863964 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:04.863834    2574 generic.go:356] "Generic (PLEG): container finished" podID="c73834fc-41c8-4eea-a739-52d16535890d" containerID="3d76081ad7dc212666e781b66847eb24565806762844d2c27e990f5ad43890cb" exitCode=0
Nov 04 01:55:04.863964 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:04.863908    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-78wz6" event={"ID":"c73834fc-41c8-4eea-a739-52d16535890d","Type":"ContainerDied","Data":"3d76081ad7dc212666e781b66847eb24565806762844d2c27e990f5ad43890cb"}
Nov 04 01:55:04.984963 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:04.984772    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2s59r" podStartSLOduration=3.05812626 podStartE2EDuration="11.984757372s" totalImagesPullingTime="8.926631112s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:04.9841504 +0000 UTC m=+13.706185860" watchObservedRunningTime="2026-11-04 01:55:04.984757372 +0000 UTC m=+13.706792834"
Nov 04 01:55:05.739565 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:05.739490    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:05.739565 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:05.739515    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:05.739565 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:05.739554    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:05.740042 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:05.739675    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74j82" podUID="1536e272-39b6-4702-a35d-0130eef3e290"
Nov 04 01:55:05.740042 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:05.739708    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g6xmn" podUID="213808a8-b7ec-4fc6-845b-d2648393d5bc"
Nov 04 01:55:05.740042 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:05.739795    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x48xl" podUID="a994b38f-0096-4fff-b49c-e6d99dc5a6be"
Nov 04 01:55:05.868653 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:05.868615    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64" event={"ID":"0aa5b60a-ce95-4623-8ae2-32896ca1232d","Type":"ContainerStarted","Data":"f205d2c13787996fa11a160f4b60c663a5185f6d93d4fa0c7eb0a3cd8e1dcded"}
Nov 04 01:55:05.870438 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:05.870410    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h4glm" event={"ID":"2522cd43-48a3-40d4-9f8f-b7a0f778abbd","Type":"ContainerStarted","Data":"230ae2dd141308db72f1397eb0fdb0ea11fd05bb4ab30fffbd9eebc5af0c924c"}
Nov 04 01:55:06.111584 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:06.111484    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64" podStartSLOduration=2.207749819 podStartE2EDuration="13.111463939s" totalImagesPullingTime="10.90371412s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:05.990423523 +0000 UTC m=+14.712458984" watchObservedRunningTime="2026-11-04 01:55:06.111463939 +0000 UTC m=+14.833499402"
Nov 04 01:55:06.112113 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:06.112084    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-h4glm" podStartSLOduration=1.759821284 podStartE2EDuration="13.112077001s" totalImagesPullingTime="11.352255717s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:06.111189224 +0000 UTC m=+14.833224686" watchObservedRunningTime="2026-11-04 01:55:06.112077001 +0000 UTC m=+14.834112462"
Nov 04 01:55:06.874764 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:06.874671    2574 generic.go:356] "Generic (PLEG): container finished" podID="c73834fc-41c8-4eea-a739-52d16535890d" containerID="9441dbe372b048a5224e8a880cf3857f91e630836a9539a2b4ffcda697c43b1d" exitCode=0
Nov 04 01:55:06.875423 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:06.874767    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-78wz6" event={"ID":"c73834fc-41c8-4eea-a739-52d16535890d","Type":"ContainerDied","Data":"9441dbe372b048a5224e8a880cf3857f91e630836a9539a2b4ffcda697c43b1d"}
Nov 04 01:55:07.739179 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:07.739150    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:07.739364 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:07.739219    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:07.739364 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:07.739315    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g6xmn" podUID="213808a8-b7ec-4fc6-845b-d2648393d5bc"
Nov 04 01:55:07.739442 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:07.739216    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:07.739442 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:07.739392    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x48xl" podUID="a994b38f-0096-4fff-b49c-e6d99dc5a6be"
Nov 04 01:55:07.739511 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:07.739477    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74j82" podUID="1536e272-39b6-4702-a35d-0130eef3e290"
Nov 04 01:55:08.881637 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:08.881598    2574 generic.go:356] "Generic (PLEG): container finished" podID="c73834fc-41c8-4eea-a739-52d16535890d" containerID="286be477883e536d2fd7ac3e03f29225f825bcbe340b5a4ccf70e28612b7c8b5" exitCode=0
Nov 04 01:55:08.882305 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:08.881653    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-78wz6" event={"ID":"c73834fc-41c8-4eea-a739-52d16535890d","Type":"ContainerDied","Data":"286be477883e536d2fd7ac3e03f29225f825bcbe340b5a4ccf70e28612b7c8b5"}
Nov 04 01:55:09.511030 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:09.510990    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:09.511256 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:09.511134    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:55:09.511256 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:09.511195    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret podName:a994b38f-0096-4fff-b49c-e6d99dc5a6be nodeName:}" failed. No retries permitted until 2026-11-04 01:55:25.511176137 +0000 UTC m=+34.233211596 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret") pod "global-pull-secret-syncer-x48xl" (UID: "a994b38f-0096-4fff-b49c-e6d99dc5a6be") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:55:09.611305 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:09.611258    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs\") pod \"network-metrics-daemon-74j82\" (UID: \"1536e272-39b6-4702-a35d-0130eef3e290\") " pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:09.611483 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:09.611334    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56z2g\" (UniqueName: \"kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g\") pod \"network-check-target-g6xmn\" (UID: \"213808a8-b7ec-4fc6-845b-d2648393d5bc\") " pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:09.611483 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:09.611433    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:55:09.611483 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:09.611465    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:55:09.611602 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:09.611487    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:55:09.611602 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:09.611500    2574 projected.go:196] Error preparing data for projected volume kube-api-access-56z2g for pod openshift-network-diagnostics/network-check-target-g6xmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:55:09.611602 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:09.611515    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs podName:1536e272-39b6-4702-a35d-0130eef3e290 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:25.611490617 +0000 UTC m=+34.333526068 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs") pod "network-metrics-daemon-74j82" (UID: "1536e272-39b6-4702-a35d-0130eef3e290") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:55:09.611602 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:09.611548    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g podName:213808a8-b7ec-4fc6-845b-d2648393d5bc nodeName:}" failed. No retries permitted until 2026-11-04 01:55:25.611532307 +0000 UTC m=+34.333567762 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-56z2g" (UniqueName: "kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g") pod "network-check-target-g6xmn" (UID: "213808a8-b7ec-4fc6-845b-d2648393d5bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:55:09.738856 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:09.738824    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:09.739011 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:09.738824    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:09.739011 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:09.738983    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g6xmn" podUID="213808a8-b7ec-4fc6-845b-d2648393d5bc"
Nov 04 01:55:09.739125 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:09.739028    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:09.739169 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:09.739124    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74j82" podUID="1536e272-39b6-4702-a35d-0130eef3e290"
Nov 04 01:55:09.739220 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:09.739186    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x48xl" podUID="a994b38f-0096-4fff-b49c-e6d99dc5a6be"
Nov 04 01:55:10.889277 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:10.889041    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph" event={"ID":"b2a74b29-e9ea-4ab9-b155-795d08684fe6","Type":"ContainerStarted","Data":"5703b6d16ad0bb4a8ae9def66f6835b5df610966e078360b37e5684600e3cc0d"}
Nov 04 01:55:10.889724 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:10.889294    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph" event={"ID":"b2a74b29-e9ea-4ab9-b155-795d08684fe6","Type":"ContainerStarted","Data":"ff9f05b010cac6fd8ec6a2eda2840b835587734a9d43e394f0395ba5d7199d92"}
Nov 04 01:55:10.889724 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:10.889337    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph" event={"ID":"b2a74b29-e9ea-4ab9-b155-795d08684fe6","Type":"ContainerStarted","Data":"e98bdba785cdc2571f69408f711f5cf4c93c104980c32ba343ce4b94ab27b460"}
Nov 04 01:55:10.889724 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:10.889385    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph" event={"ID":"b2a74b29-e9ea-4ab9-b155-795d08684fe6","Type":"ContainerStarted","Data":"5849609fac2bd907a3371413c78ddb8baf4901bfca10343b4349d6607fca434b"}
Nov 04 01:55:10.889724 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:10.889428    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph" event={"ID":"b2a74b29-e9ea-4ab9-b155-795d08684fe6","Type":"ContainerStarted","Data":"c3291d499930aeac645de2058e889dfde622726fb97f357e1c415b2bbce64e75"}
Nov 04 01:55:10.889724 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:10.889455    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph" event={"ID":"b2a74b29-e9ea-4ab9-b155-795d08684fe6","Type":"ContainerStarted","Data":"4cee66c691b2b98fe333d4a61c57821cc7075bc24efef1d397a64b76f54b65d5"}
Nov 04 01:55:11.739542 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:11.739510    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:11.739729 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:11.739607    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74j82" podUID="1536e272-39b6-4702-a35d-0130eef3e290"
Nov 04 01:55:11.739729 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:11.739651    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:11.739729 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:11.739730    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:11.739868 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:11.739749    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x48xl" podUID="a994b38f-0096-4fff-b49c-e6d99dc5a6be"
Nov 04 01:55:11.739868 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:11.739807    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g6xmn" podUID="213808a8-b7ec-4fc6-845b-d2648393d5bc"
Nov 04 01:55:11.893888 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:11.893861    2574 generic.go:356] "Generic (PLEG): container finished" podID="c73834fc-41c8-4eea-a739-52d16535890d" containerID="90c64767df1ba08401c4b652a3e18f845d6c7c8b98bd0e81a08fd0b65ccd1bbe" exitCode=0
Nov 04 01:55:11.894301 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:11.893909    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-78wz6" event={"ID":"c73834fc-41c8-4eea-a739-52d16535890d","Type":"ContainerDied","Data":"90c64767df1ba08401c4b652a3e18f845d6c7c8b98bd0e81a08fd0b65ccd1bbe"}
Nov 04 01:55:12.898470 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:12.898386    2574 generic.go:356] "Generic (PLEG): container finished" podID="c73834fc-41c8-4eea-a739-52d16535890d" containerID="8eb378ffb472b8adb4b21b30a7efe31c8601eb082671d06647339fa55d3d2dac" exitCode=0
Nov 04 01:55:12.898988 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:12.898479    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-78wz6" event={"ID":"c73834fc-41c8-4eea-a739-52d16535890d","Type":"ContainerDied","Data":"8eb378ffb472b8adb4b21b30a7efe31c8601eb082671d06647339fa55d3d2dac"}
Nov 04 01:55:12.901814 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:12.901788    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph" event={"ID":"b2a74b29-e9ea-4ab9-b155-795d08684fe6","Type":"ContainerStarted","Data":"511613891fd8c00d61dac558443b791e2e2109fdcf2c9a1c379febdd513feb55"}
Nov 04 01:55:13.739103 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:13.739059    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:13.739103 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:13.739082    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:13.739275 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:13.739179    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:13.739275 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:13.739194    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74j82" podUID="1536e272-39b6-4702-a35d-0130eef3e290"
Nov 04 01:55:13.739361 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:13.739300    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x48xl" podUID="a994b38f-0096-4fff-b49c-e6d99dc5a6be"
Nov 04 01:55:13.739420 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:13.739373    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g6xmn" podUID="213808a8-b7ec-4fc6-845b-d2648393d5bc"
Nov 04 01:55:13.814337 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:13.814302    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dszxl"
Nov 04 01:55:13.814481 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:13.814426    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:55:13.814819 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:13.814806    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dszxl"
Nov 04 01:55:13.906904 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:13.906871    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-78wz6" event={"ID":"c73834fc-41c8-4eea-a739-52d16535890d","Type":"ContainerStarted","Data":"d468fa952b77a0d4be6b3fac5e89170544892459cc19dcbf068f83768a7355eb"}
Nov 04 01:55:14.054445 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:14.054342    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-78wz6" podStartSLOduration=7.408753743 podStartE2EDuration="21.054328948s" totalImagesPullingTime="13.645575205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:53 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:14.053861729 +0000 UTC m=+22.775897191" watchObservedRunningTime="2026-11-04 01:55:14.054328948 +0000 UTC m=+22.776364409"
Nov 04 01:55:14.914259 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:14.914217    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph" event={"ID":"b2a74b29-e9ea-4ab9-b155-795d08684fe6","Type":"ContainerStarted","Data":"07c82edd440b25b289cf55c0d3d12180d9d3b7d599b6123af030a25db21cfcfd"}
Nov 04 01:55:15.195011 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:15.194965    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph" podStartSLOduration=6.066118842 podStartE2EDuration="22.194951455s" totalImagesPullingTime="16.128832613s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:15.194891162 +0000 UTC m=+23.916926622" watchObservedRunningTime="2026-11-04 01:55:15.194951455 +0000 UTC m=+23.916986917"
Nov 04 01:55:15.738966 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:15.738934    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:15.739201 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:15.738940    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:15.739201 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:15.739060    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x48xl" podUID="a994b38f-0096-4fff-b49c-e6d99dc5a6be"
Nov 04 01:55:15.739201 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:15.738952    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:15.739201 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:15.739124    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g6xmn" podUID="213808a8-b7ec-4fc6-845b-d2648393d5bc"
Nov 04 01:55:15.739333 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:15.739235    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74j82" podUID="1536e272-39b6-4702-a35d-0130eef3e290"
Nov 04 01:55:15.922409 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:15.919658    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:55:15.922409 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:15.920591    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:15.922409 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:15.921127    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:15.944072 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:15.943970    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:15.945178 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:15.945152    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:16.921745 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:16.921715    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:55:17.450422 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:17.450230    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x48xl"]
Nov 04 01:55:17.450782 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:17.450501    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:17.450782 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:17.450582    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x48xl" podUID="a994b38f-0096-4fff-b49c-e6d99dc5a6be"
Nov 04 01:55:17.453407 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:17.453370    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g6xmn"]
Nov 04 01:55:17.453540 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:17.453477    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:17.453604 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:17.453555    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g6xmn" podUID="213808a8-b7ec-4fc6-845b-d2648393d5bc"
Nov 04 01:55:17.456122 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:17.456100    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-74j82"]
Nov 04 01:55:17.456226 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:17.456197    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:17.456320 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:17.456300    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74j82" podUID="1536e272-39b6-4702-a35d-0130eef3e290"
Nov 04 01:55:17.924385 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:17.924315    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:55:18.739654 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:18.739624    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:18.740025 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:18.739723    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g6xmn" podUID="213808a8-b7ec-4fc6-845b-d2648393d5bc"
Nov 04 01:55:19.739863 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:19.739780    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:19.739863 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:19.739805    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:19.740458 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:19.739895    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74j82" podUID="1536e272-39b6-4702-a35d-0130eef3e290"
Nov 04 01:55:19.740458 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:19.740037    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x48xl" podUID="a994b38f-0096-4fff-b49c-e6d99dc5a6be"
Nov 04 01:55:20.182037 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:20.181963    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:20.182256 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:20.182166    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:55:20.192938 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:20.192897    2574 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph" podUID="b2a74b29-e9ea-4ab9-b155-795d08684fe6" containerName="ovnkube-controller" probeResult="failure" output=""
Nov 04 01:55:20.200843 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:20.200821    2574 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph" podUID="b2a74b29-e9ea-4ab9-b155-795d08684fe6" containerName="ovnkube-controller" probeResult="failure" output=""
Nov 04 01:55:20.739234 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:20.739186    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:20.739460 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:20.739325    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g6xmn" podUID="213808a8-b7ec-4fc6-845b-d2648393d5bc"
Nov 04 01:55:21.741776 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:21.741746    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:21.742166 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:21.741747    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:21.742166 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:21.741881    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74j82" podUID="1536e272-39b6-4702-a35d-0130eef3e290"
Nov 04 01:55:21.742166 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:21.741944    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x48xl" podUID="a994b38f-0096-4fff-b49c-e6d99dc5a6be"
Nov 04 01:55:22.592569 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.592533    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeReady"
Nov 04 01:55:22.592726 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.592685    2574 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 04 01:55:22.708694 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.708661    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-rzbgn"]
Nov 04 01:55:22.711506 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.711490    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-rzbgn"
Nov 04 01:55:22.739208 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.739183    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:22.747685 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.747666    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-p48wv"]
Nov 04 01:55:22.750716 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.750684    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"]
Nov 04 01:55:22.750862 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.750841    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-p48wv"
Nov 04 01:55:22.753807 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.753789    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-7gmmg"]
Nov 04 01:55:22.754342 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.754323    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"
Nov 04 01:55:22.756976 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.756959    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-7gmmg"
Nov 04 01:55:22.759384 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.759366    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"]
Nov 04 01:55:22.762361 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.762344    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"]
Nov 04 01:55:22.763137 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.763117    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:22.765343 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.765325    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8574f7698d-qmq69"]
Nov 04 01:55:22.765936 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.765919    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:22.768326 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.768308    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"]
Nov 04 01:55:22.768515 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.768493    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:22.771803 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.771654    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-2bzkj"]
Nov 04 01:55:22.771803 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.771788    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:22.775434 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.775415    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-56bdbb678c-mvkmq"]
Nov 04 01:55:22.775570 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.775552    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:22.778879 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.778837    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:22.799698 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.798831    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"]
Nov 04 01:55:22.801840 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.801821    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-cw5p4"]
Nov 04 01:55:22.801972 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.801953    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"
Nov 04 01:55:22.804630 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.804613    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c59b47685-9jmnr"]
Nov 04 01:55:22.804728 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.804714    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-cw5p4"
Nov 04 01:55:22.807380 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.807364    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-p2kf6"]
Nov 04 01:55:22.807470 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.807422    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:22.810255 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.810216    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv54m\" (UniqueName: \"kubernetes.io/projected/c07ce523-d075-4203-be44-3f7d5af7e0d7-kube-api-access-xv54m\") pod \"kas-connection-checker-685895c7c9-rzbgn\" (UID: \"c07ce523-d075-4203-be44-3f7d5af7e0d7\") " pod="kube-system/kas-connection-checker-685895c7c9-rzbgn"
Nov 04 01:55:22.811246 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.811146    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 04 01:55:22.811745 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.811726    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 04 01:55:22.815436 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.812667    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zzzll\""
Nov 04 01:55:22.815436 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.814643    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-p2kf6"
Nov 04 01:55:22.815718 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.815699    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"]
Nov 04 01:55:22.818694 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.818671    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-68kbx"]
Nov 04 01:55:22.818887 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.818870    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:22.822124 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.822084    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5ljw9"]
Nov 04 01:55:22.822274 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.822188    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:22.825395 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.825377    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-rzbgn"]
Nov 04 01:55:22.825534 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.825411    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-p48wv"]
Nov 04 01:55:22.825534 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.825435    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-7gmmg"]
Nov 04 01:55:22.825534 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.825454    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"]
Nov 04 01:55:22.825534 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.825472    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"]
Nov 04 01:55:22.825534 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.825496    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"]
Nov 04 01:55:22.825534 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.825515    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"]
Nov 04 01:55:22.825534 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.825533    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8574f7698d-qmq69"]
Nov 04 01:55:22.825971 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.825552    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-2bzkj"]
Nov 04 01:55:22.825971 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.825571    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-68kbx"]
Nov 04 01:55:22.825971 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.825646    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5ljw9"
Nov 04 01:55:22.849580 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.849551    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-p2kf6"]
Nov 04 01:55:22.850210 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.850189    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c59b47685-9jmnr"]
Nov 04 01:55:22.850759 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.850743    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"]
Nov 04 01:55:22.851266 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.851246    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-cw5p4"]
Nov 04 01:55:22.856134 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.856111    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"]
Nov 04 01:55:22.860964 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.860501    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-56bdbb678c-mvkmq"]
Nov 04 01:55:22.861798 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.861777    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5ljw9"]
Nov 04 01:55:22.866413 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.866379    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 01:55:22.867379 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.867360    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 04 01:55:22.867475 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.867400    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:55:22.867475 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.867412    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 04 01:55:22.867475 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.867464    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 04 01:55:22.867921 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.867903    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 04 01:55:22.868000 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.867925    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 01:55:22.868000 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.867945    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 01:55:22.868000 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.867964    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 04 01:55:22.868000 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.867983    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 04 01:55:22.868000 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.867913    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:55:22.868234 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.868042    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 04 01:55:22.868234 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.868062    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 04 01:55:22.868234 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.868110    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 04 01:55:22.868234 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.868123    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 01:55:22.868234 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.868160    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 04 01:55:22.868639 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.868622    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:55:22.868721 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.868625    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 04 01:55:22.868721 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.868653    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:55:22.868721 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.868712    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 01:55:22.869009 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.868716    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 04 01:55:22.899453 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.899429    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:55:22.899563 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.899539    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 01:55:22.911248 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911228    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:22.911248 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911255    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c63f4e36-582d-43be-8575-3b79b7c4756d-nginx-conf\") pod \"networking-console-plugin-86c478bb88-8rvcg\" (UID: \"c63f4e36-582d-43be-8575-3b79b7c4756d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:22.911442 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911301    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-snapshots\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:22.911442 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911348    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrt5f\" (UniqueName: \"kubernetes.io/projected/ef75aae9-bb5f-4e09-b374-fa398aadfae9-kube-api-access-mrt5f\") pod \"kas-connection-checker-685895c7c9-p48wv\" (UID: \"ef75aae9-bb5f-4e09-b374-fa398aadfae9\") " pod="kube-system/kas-connection-checker-685895c7c9-p48wv"
Nov 04 01:55:22.911442 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911380    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-image-registry-private-configuration\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:22.911442 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911408    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv54m\" (UniqueName: \"kubernetes.io/projected/c07ce523-d075-4203-be44-3f7d5af7e0d7-kube-api-access-xv54m\") pod \"kas-connection-checker-685895c7c9-rzbgn\" (UID: \"c07ce523-d075-4203-be44-3f7d5af7e0d7\") " pod="kube-system/kas-connection-checker-685895c7c9-rzbgn"
Nov 04 01:55:22.911442 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911424    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:22.911442 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911441    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/417744b4-5ed7-4b20-9960-7147d2d567ee-trusted-ca\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:22.911643 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911458    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kslbj\" (UniqueName: \"kubernetes.io/projected/d09416ce-b095-4a28-9cb9-c0ad486ec03c-kube-api-access-kslbj\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-84j4p\" (UID: \"d09416ce-b095-4a28-9cb9-c0ad486ec03c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"
Nov 04 01:55:22.911643 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911517    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:22.911643 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911557    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-certificates\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:22.911643 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911615    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcz47\" (UniqueName: \"kubernetes.io/projected/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-kube-api-access-mcz47\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:22.911802 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911692    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/698e630d-a5b0-43ce-9a12-aac9c60ab53c-serving-cert\") pod \"service-ca-operator-544d749c8c-wzh4t\" (UID: \"698e630d-a5b0-43ce-9a12-aac9c60ab53c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"
Nov 04 01:55:22.911802 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911756    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f68tp\" (UniqueName: \"kubernetes.io/projected/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-kube-api-access-f68tp\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:22.911802 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911777    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:22.911895 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911836    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/74e217e6-636d-4bab-b37f-c44dc0d8fe81-ca-trust-extracted\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:22.911895 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911877    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-stats-auth\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:22.911974 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911903    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-registry-certificates\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:22.911974 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911923    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-bound-sa-token\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:22.911974 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.911939    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:22.912127 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912026    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d09416ce-b095-4a28-9cb9-c0ad486ec03c-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-84j4p\" (UID: \"d09416ce-b095-4a28-9cb9-c0ad486ec03c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"
Nov 04 01:55:22.912127 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912058    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/698e630d-a5b0-43ce-9a12-aac9c60ab53c-config\") pod \"service-ca-operator-544d749c8c-wzh4t\" (UID: \"698e630d-a5b0-43ce-9a12-aac9c60ab53c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"
Nov 04 01:55:22.912127 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912086    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fppqt\" (UniqueName: \"kubernetes.io/projected/698e630d-a5b0-43ce-9a12-aac9c60ab53c-kube-api-access-fppqt\") pod \"service-ca-operator-544d749c8c-wzh4t\" (UID: \"698e630d-a5b0-43ce-9a12-aac9c60ab53c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"
Nov 04 01:55:22.912127 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912112    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:22.912245 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912154    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-installation-pull-secrets\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:22.912245 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912194    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b655g\" (UniqueName: \"kubernetes.io/projected/40363d65-7ece-4f05-8668-439a60582a18-kube-api-access-b655g\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:22.912245 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912213    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-serving-cert\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:22.912245 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912233    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-bound-sa-token\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:22.912396 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912249    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44d8q\" (UniqueName: \"kubernetes.io/projected/b461004a-433c-4411-84df-b9aea9f682ab-kube-api-access-44d8q\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:22.912396 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912268    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmpkm\" (UniqueName: \"kubernetes.io/projected/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-kube-api-access-rmpkm\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:22.912396 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912334    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c8q8\" (UniqueName: \"kubernetes.io/projected/1c13ee40-c34c-4575-8865-aceeb71759d0-kube-api-access-2c8q8\") pod \"volume-data-source-validator-58c9f5f46d-cw5p4\" (UID: \"1c13ee40-c34c-4575-8865-aceeb71759d0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-cw5p4"
Nov 04 01:55:22.912396 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912351    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b461004a-433c-4411-84df-b9aea9f682ab-tmp-dir\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:22.912396 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912366    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-8rvcg\" (UID: \"c63f4e36-582d-43be-8575-3b79b7c4756d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:22.912602 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912402    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:22.912602 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912511    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3db2db6-6f0f-4af7-9bb7-493df9107bb9-cert\") pod \"ingress-canary-5ljw9\" (UID: \"f3db2db6-6f0f-4af7-9bb7-493df9107bb9\") " pod="openshift-ingress-canary/ingress-canary-5ljw9"
Nov 04 01:55:22.912602 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912541    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xrfz\" (UniqueName: \"kubernetes.io/projected/f3db2db6-6f0f-4af7-9bb7-493df9107bb9-kube-api-access-2xrfz\") pod \"ingress-canary-5ljw9\" (UID: \"f3db2db6-6f0f-4af7-9bb7-493df9107bb9\") " pod="openshift-ingress-canary/ingress-canary-5ljw9"
Nov 04 01:55:22.912602 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912567    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2c2n\" (UniqueName: \"kubernetes.io/projected/4d14f606-6852-4eff-b71f-5a8adcde5f84-kube-api-access-v2c2n\") pod \"network-check-source-6f78d59f99-p2kf6\" (UID: \"4d14f606-6852-4eff-b71f-5a8adcde5f84\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-p2kf6"
Nov 04 01:55:22.912766 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912627    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8qmb\" (UniqueName: \"kubernetes.io/projected/99ef12a9-a676-4258-a3a8-e7fcd611d158-kube-api-access-k8qmb\") pod \"kas-connection-checker-685895c7c9-7gmmg\" (UID: \"99ef12a9-a676-4258-a3a8-e7fcd611d158\") " pod="kube-system/kas-connection-checker-685895c7c9-7gmmg"
Nov 04 01:55:22.912766 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912655    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/417744b4-5ed7-4b20-9960-7147d2d567ee-config\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:22.912766 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912701    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/417744b4-5ed7-4b20-9960-7147d2d567ee-serving-cert\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:22.912886 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912765    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-image-registry-private-configuration\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:22.912886 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912792    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d09416ce-b095-4a28-9cb9-c0ad486ec03c-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-84j4p\" (UID: \"d09416ce-b095-4a28-9cb9-c0ad486ec03c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"
Nov 04 01:55:22.912886 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912806    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/40363d65-7ece-4f05-8668-439a60582a18-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:22.912886 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912871    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-trusted-ca\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:22.913028 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912901    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b461004a-433c-4411-84df-b9aea9f682ab-config-volume\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:22.913028 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912929    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-tmp\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:22.913028 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912961    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-service-ca-bundle\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:22.913028 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.912986    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-installation-pull-secrets\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:22.913028 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.913017    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9hwp\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-kube-api-access-k9hwp\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:22.913165 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.913034    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-trusted-ca\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:22.913165 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.913073    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:22.913165 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.913101    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-default-certificate\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:22.913165 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.913136    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:22.913165 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.913153    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlzhr\" (UniqueName: \"kubernetes.io/projected/417744b4-5ed7-4b20-9960-7147d2d567ee-kube-api-access-nlzhr\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:22.913334 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.913182    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g6rr\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-kube-api-access-9g6rr\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:22.913334 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.913210    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/870962af-eac0-484d-9267-db8ab42ef13d-ca-trust-extracted\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:22.919948 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.919919    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 04 01:55:22.920048 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.920006    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 04 01:55:22.920112 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.920097    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 04 01:55:22.920231 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.920217    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 04 01:55:22.920348 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.920300    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 04 01:55:22.920394 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.920366    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 04 01:55:22.920434 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.920427    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 04 01:55:22.922080 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.922056    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 04 01:55:22.922303 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.922270    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 01:55:22.922510 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.922494    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 04 01:55:22.922617 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.922599    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 01:55:22.922802 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.922788    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 04 01:55:22.922961 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.922948    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 04 01:55:22.923098 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.922376    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k89rs\""
Nov 04 01:55:22.923098 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.922495    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-9ldb4\""
Nov 04 01:55:22.923269 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.923119    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 04 01:55:22.923269 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.923132    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 04 01:55:22.923269 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.923196    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 01:55:22.923269 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.923250    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 04 01:55:22.923510 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.923338    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 04 01:55:22.923510 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.923357    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-qmgtz\""
Nov 04 01:55:22.923510 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.923436    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zsfkj\""
Nov 04 01:55:22.950705 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.950687    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 04 01:55:22.965833 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.965802    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 01:55:22.965946 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.965878    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 01:55:22.965946 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.965879    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 01:55:22.965946 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.965935    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 01:55:22.966359 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.966341    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 04 01:55:22.974318 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:22.974298    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv54m\" (UniqueName: \"kubernetes.io/projected/c07ce523-d075-4203-be44-3f7d5af7e0d7-kube-api-access-xv54m\") pod \"kas-connection-checker-685895c7c9-rzbgn\" (UID: \"c07ce523-d075-4203-be44-3f7d5af7e0d7\") " pod="kube-system/kas-connection-checker-685895c7c9-rzbgn"
Nov 04 01:55:23.013754 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.013725    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlzhr\" (UniqueName: \"kubernetes.io/projected/417744b4-5ed7-4b20-9960-7147d2d567ee-kube-api-access-nlzhr\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:23.013754 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.013756    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g6rr\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-kube-api-access-9g6rr\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:23.013900 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.013782    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/870962af-eac0-484d-9267-db8ab42ef13d-ca-trust-extracted\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:23.013900 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.013807    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:23.013900 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.013834    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c63f4e36-582d-43be-8575-3b79b7c4756d-nginx-conf\") pod \"networking-console-plugin-86c478bb88-8rvcg\" (UID: \"c63f4e36-582d-43be-8575-3b79b7c4756d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:23.013900 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.013870    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-snapshots\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:23.013900 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.013896    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrt5f\" (UniqueName: \"kubernetes.io/projected/ef75aae9-bb5f-4e09-b374-fa398aadfae9-kube-api-access-mrt5f\") pod \"kas-connection-checker-685895c7c9-p48wv\" (UID: \"ef75aae9-bb5f-4e09-b374-fa398aadfae9\") " pod="kube-system/kas-connection-checker-685895c7c9-p48wv"
Nov 04 01:55:23.014134 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.013921    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-image-registry-private-configuration\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:23.014134 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.013948    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:23.014134 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.013974    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/417744b4-5ed7-4b20-9960-7147d2d567ee-trusted-ca\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:23.014134 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.013981    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:55:23.014134 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014007    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kslbj\" (UniqueName: \"kubernetes.io/projected/d09416ce-b095-4a28-9cb9-c0ad486ec03c-kube-api-access-kslbj\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-84j4p\" (UID: \"d09416ce-b095-4a28-9cb9-c0ad486ec03c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"
Nov 04 01:55:23.014134 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014039    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:23.014134 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.014063    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs podName:cc72c8e1-7e7c-4f53-946b-6ef94d6c5971 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:23.514042807 +0000 UTC m=+32.236078249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs") pod "router-default-56bdbb678c-mvkmq" (UID: "cc72c8e1-7e7c-4f53-946b-6ef94d6c5971") : secret "router-metrics-certs-default" not found
Nov 04 01:55:23.014134 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014101    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-certificates\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:23.014134 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.014115    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:23.014134 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014128    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcz47\" (UniqueName: \"kubernetes.io/projected/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-kube-api-access-mcz47\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:23.014667 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014155    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/698e630d-a5b0-43ce-9a12-aac9c60ab53c-serving-cert\") pod \"service-ca-operator-544d749c8c-wzh4t\" (UID: \"698e630d-a5b0-43ce-9a12-aac9c60ab53c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"
Nov 04 01:55:23.014667 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014180    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f68tp\" (UniqueName: \"kubernetes.io/projected/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-kube-api-access-f68tp\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:23.014667 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014207    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:23.014667 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.014220    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls podName:40363d65-7ece-4f05-8668-439a60582a18 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:23.514202741 +0000 UTC m=+32.236238182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wzmdz" (UID: "40363d65-7ece-4f05-8668-439a60582a18") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:23.014667 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014254    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/74e217e6-636d-4bab-b37f-c44dc0d8fe81-ca-trust-extracted\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:23.014667 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014307    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-stats-auth\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:23.014667 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014336    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-registry-certificates\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:23.014667 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014366    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-bound-sa-token\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:23.014667 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014390    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:23.014667 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014418    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d09416ce-b095-4a28-9cb9-c0ad486ec03c-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-84j4p\" (UID: \"d09416ce-b095-4a28-9cb9-c0ad486ec03c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"
Nov 04 01:55:23.014667 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014443    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/698e630d-a5b0-43ce-9a12-aac9c60ab53c-config\") pod \"service-ca-operator-544d749c8c-wzh4t\" (UID: \"698e630d-a5b0-43ce-9a12-aac9c60ab53c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"
Nov 04 01:55:23.014667 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014472    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fppqt\" (UniqueName: \"kubernetes.io/projected/698e630d-a5b0-43ce-9a12-aac9c60ab53c-kube-api-access-fppqt\") pod \"service-ca-operator-544d749c8c-wzh4t\" (UID: \"698e630d-a5b0-43ce-9a12-aac9c60ab53c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"
Nov 04 01:55:23.014667 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014500    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:23.014667 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014541    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-installation-pull-secrets\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:23.014667 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014571    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b655g\" (UniqueName: \"kubernetes.io/projected/40363d65-7ece-4f05-8668-439a60582a18-kube-api-access-b655g\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:23.015391 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014597    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-serving-cert\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:23.015391 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014619    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-bound-sa-token\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:23.015391 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014653    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44d8q\" (UniqueName: \"kubernetes.io/projected/b461004a-433c-4411-84df-b9aea9f682ab-kube-api-access-44d8q\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:23.015391 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014685    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmpkm\" (UniqueName: \"kubernetes.io/projected/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-kube-api-access-rmpkm\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:23.015391 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014712    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c8q8\" (UniqueName: \"kubernetes.io/projected/1c13ee40-c34c-4575-8865-aceeb71759d0-kube-api-access-2c8q8\") pod \"volume-data-source-validator-58c9f5f46d-cw5p4\" (UID: \"1c13ee40-c34c-4575-8865-aceeb71759d0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-cw5p4"
Nov 04 01:55:23.015391 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014744    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b461004a-433c-4411-84df-b9aea9f682ab-tmp-dir\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:23.015391 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014780    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-8rvcg\" (UID: \"c63f4e36-582d-43be-8575-3b79b7c4756d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:23.015391 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014805    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:23.015391 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014847    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3db2db6-6f0f-4af7-9bb7-493df9107bb9-cert\") pod \"ingress-canary-5ljw9\" (UID: \"f3db2db6-6f0f-4af7-9bb7-493df9107bb9\") " pod="openshift-ingress-canary/ingress-canary-5ljw9"
Nov 04 01:55:23.015391 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014871    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xrfz\" (UniqueName: \"kubernetes.io/projected/f3db2db6-6f0f-4af7-9bb7-493df9107bb9-kube-api-access-2xrfz\") pod \"ingress-canary-5ljw9\" (UID: \"f3db2db6-6f0f-4af7-9bb7-493df9107bb9\") " pod="openshift-ingress-canary/ingress-canary-5ljw9"
Nov 04 01:55:23.015391 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014900    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2c2n\" (UniqueName: \"kubernetes.io/projected/4d14f606-6852-4eff-b71f-5a8adcde5f84-kube-api-access-v2c2n\") pod \"network-check-source-6f78d59f99-p2kf6\" (UID: \"4d14f606-6852-4eff-b71f-5a8adcde5f84\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-p2kf6"
Nov 04 01:55:23.015391 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014932    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8qmb\" (UniqueName: \"kubernetes.io/projected/99ef12a9-a676-4258-a3a8-e7fcd611d158-kube-api-access-k8qmb\") pod \"kas-connection-checker-685895c7c9-7gmmg\" (UID: \"99ef12a9-a676-4258-a3a8-e7fcd611d158\") " pod="kube-system/kas-connection-checker-685895c7c9-7gmmg"
Nov 04 01:55:23.015391 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014958    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/417744b4-5ed7-4b20-9960-7147d2d567ee-config\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:23.015391 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014982    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/417744b4-5ed7-4b20-9960-7147d2d567ee-serving-cert\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:23.015391 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014998    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-certificates\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:23.015391 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015023    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c63f4e36-582d-43be-8575-3b79b7c4756d-nginx-conf\") pod \"networking-console-plugin-86c478bb88-8rvcg\" (UID: \"c63f4e36-582d-43be-8575-3b79b7c4756d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:23.015391 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.015117    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:23.016190 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.015130    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c59b47685-9jmnr: secret "image-registry-tls" not found
Nov 04 01:55:23.016190 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.015181    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls podName:870962af-eac0-484d-9267-db8ab42ef13d nodeName:}" failed. No retries permitted until 2026-11-04 01:55:23.515167993 +0000 UTC m=+32.237203445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls") pod "image-registry-6c59b47685-9jmnr" (UID: "870962af-eac0-484d-9267-db8ab42ef13d") : secret "image-registry-tls" not found
Nov 04 01:55:23.016190 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015223    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:23.016190 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015009    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-image-registry-private-configuration\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:23.016190 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015398    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/698e630d-a5b0-43ce-9a12-aac9c60ab53c-config\") pod \"service-ca-operator-544d749c8c-wzh4t\" (UID: \"698e630d-a5b0-43ce-9a12-aac9c60ab53c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"
Nov 04 01:55:23.016190 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015425    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d09416ce-b095-4a28-9cb9-c0ad486ec03c-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-84j4p\" (UID: \"d09416ce-b095-4a28-9cb9-c0ad486ec03c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"
Nov 04 01:55:23.016190 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.015445    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle podName:cc72c8e1-7e7c-4f53-946b-6ef94d6c5971 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:23.515429983 +0000 UTC m=+32.237465426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle") pod "router-default-56bdbb678c-mvkmq" (UID: "cc72c8e1-7e7c-4f53-946b-6ef94d6c5971") : configmap references non-existent config key: service-ca.crt
Nov 04 01:55:23.016190 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015476    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/40363d65-7ece-4f05-8668-439a60582a18-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:23.016190 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015504    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-trusted-ca\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:23.016190 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015514    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-registry-certificates\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:23.016190 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015528    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b461004a-433c-4411-84df-b9aea9f682ab-config-volume\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:23.016190 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015567    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-tmp\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:23.016190 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015594    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-service-ca-bundle\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:23.016190 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015622    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-installation-pull-secrets\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:23.016190 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015647    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9hwp\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-kube-api-access-k9hwp\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:23.016929 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015681    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-trusted-ca\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:23.016929 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015721    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:23.016929 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015746    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-default-certificate\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:23.016929 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015778    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:23.016929 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.015797    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:55:23.016929 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.015843    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls podName:b461004a-433c-4411-84df-b9aea9f682ab nodeName:}" failed. No retries permitted until 2026-11-04 01:55:23.515826208 +0000 UTC m=+32.237861664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls") pod "dns-default-68kbx" (UID: "b461004a-433c-4411-84df-b9aea9f682ab") : secret "dns-default-metrics-tls" not found
Nov 04 01:55:23.016929 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.015909    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:23.016929 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.015921    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8574f7698d-qmq69: secret "image-registry-tls" not found
Nov 04 01:55:23.016929 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.015956    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls podName:74e217e6-636d-4bab-b37f-c44dc0d8fe81 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:23.515945011 +0000 UTC m=+32.237980470 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls") pod "image-registry-8574f7698d-qmq69" (UID: "74e217e6-636d-4bab-b37f-c44dc0d8fe81") : secret "image-registry-tls" not found
Nov 04 01:55:23.016929 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.015974    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d09416ce-b095-4a28-9cb9-c0ad486ec03c-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-84j4p\" (UID: \"d09416ce-b095-4a28-9cb9-c0ad486ec03c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"
Nov 04 01:55:23.016929 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.016548    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:23.016929 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.014214    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/870962af-eac0-484d-9267-db8ab42ef13d-ca-trust-extracted\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:23.016929 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.016847    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/74e217e6-636d-4bab-b37f-c44dc0d8fe81-ca-trust-extracted\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:23.017530 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.016934    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/417744b4-5ed7-4b20-9960-7147d2d567ee-config\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:23.017530 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.017013    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b461004a-433c-4411-84df-b9aea9f682ab-tmp-dir\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:23.017530 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.017104    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:55:23.017530 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.017149    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert podName:c63f4e36-582d-43be-8575-3b79b7c4756d nodeName:}" failed. No retries permitted until 2026-11-04 01:55:23.517135487 +0000 UTC m=+32.239170958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-8rvcg" (UID: "c63f4e36-582d-43be-8575-3b79b7c4756d") : secret "networking-console-plugin-cert" not found
Nov 04 01:55:23.017530 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.017426    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/417744b4-5ed7-4b20-9960-7147d2d567ee-trusted-ca\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:23.017530 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.017480    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-snapshots\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:23.017939 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.017909    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/40363d65-7ece-4f05-8668-439a60582a18-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:23.018093 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.018070    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d09416ce-b095-4a28-9cb9-c0ad486ec03c-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-84j4p\" (UID: \"d09416ce-b095-4a28-9cb9-c0ad486ec03c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"
Nov 04 01:55:23.018776 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.018754    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:55:23.018879 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.018808    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls podName:eddb65a3-0fa8-48b5-8ec9-4c939e189be4 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:23.518795278 +0000 UTC m=+32.240830717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-sr887" (UID: "eddb65a3-0fa8-48b5-8ec9-4c939e189be4") : secret "samples-operator-tls" not found
Nov 04 01:55:23.019076 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.019054    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-image-registry-private-configuration\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:23.019148 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.019075    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-image-registry-private-configuration\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:23.019148 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.018753    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-trusted-ca\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:23.019303 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.019262    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b461004a-433c-4411-84df-b9aea9f682ab-config-volume\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:23.019405 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.019382    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-tmp\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:23.019640 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.019611    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-service-ca-bundle\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:23.019716 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.019665    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-trusted-ca\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:23.019906 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.019878    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/698e630d-a5b0-43ce-9a12-aac9c60ab53c-serving-cert\") pod \"service-ca-operator-544d749c8c-wzh4t\" (UID: \"698e630d-a5b0-43ce-9a12-aac9c60ab53c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"
Nov 04 01:55:23.020092 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.020032    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-stats-auth\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:23.020438 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.020416    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-serving-cert\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:23.020734 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.020709    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/417744b4-5ed7-4b20-9960-7147d2d567ee-serving-cert\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:23.021032 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.021001    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-installation-pull-secrets\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:23.021326 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.021300    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-default-certificate\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:23.021876 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.021842    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3db2db6-6f0f-4af7-9bb7-493df9107bb9-cert\") pod \"ingress-canary-5ljw9\" (UID: \"f3db2db6-6f0f-4af7-9bb7-493df9107bb9\") " pod="openshift-ingress-canary/ingress-canary-5ljw9"
Nov 04 01:55:23.022093 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.022077    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-installation-pull-secrets\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:23.022976 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.022945    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-rzbgn"
Nov 04 01:55:23.079806 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.079411    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlzhr\" (UniqueName: \"kubernetes.io/projected/417744b4-5ed7-4b20-9960-7147d2d567ee-kube-api-access-nlzhr\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:23.079806 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.079748    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f68tp\" (UniqueName: \"kubernetes.io/projected/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-kube-api-access-f68tp\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:23.079806 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.079777    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcz47\" (UniqueName: \"kubernetes.io/projected/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-kube-api-access-mcz47\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:23.080237 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.080189    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g6rr\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-kube-api-access-9g6rr\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:23.081388 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.081326    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrt5f\" (UniqueName: \"kubernetes.io/projected/ef75aae9-bb5f-4e09-b374-fa398aadfae9-kube-api-access-mrt5f\") pod \"kas-connection-checker-685895c7c9-p48wv\" (UID: \"ef75aae9-bb5f-4e09-b374-fa398aadfae9\") " pod="kube-system/kas-connection-checker-685895c7c9-p48wv"
Nov 04 01:55:23.083337 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.083205    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-bound-sa-token\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:23.083337 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.083224    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fppqt\" (UniqueName: \"kubernetes.io/projected/698e630d-a5b0-43ce-9a12-aac9c60ab53c-kube-api-access-fppqt\") pod \"service-ca-operator-544d749c8c-wzh4t\" (UID: \"698e630d-a5b0-43ce-9a12-aac9c60ab53c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"
Nov 04 01:55:23.084219 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.084197    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xrfz\" (UniqueName: \"kubernetes.io/projected/f3db2db6-6f0f-4af7-9bb7-493df9107bb9-kube-api-access-2xrfz\") pod \"ingress-canary-5ljw9\" (UID: \"f3db2db6-6f0f-4af7-9bb7-493df9107bb9\") " pod="openshift-ingress-canary/ingress-canary-5ljw9"
Nov 04 01:55:23.085164 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.085110    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-bound-sa-token\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:23.085873 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.085847    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2c2n\" (UniqueName: \"kubernetes.io/projected/4d14f606-6852-4eff-b71f-5a8adcde5f84-kube-api-access-v2c2n\") pod \"network-check-source-6f78d59f99-p2kf6\" (UID: \"4d14f606-6852-4eff-b71f-5a8adcde5f84\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-p2kf6"
Nov 04 01:55:23.086005 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.085907    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmpkm\" (UniqueName: \"kubernetes.io/projected/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-kube-api-access-rmpkm\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:23.086336 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.086258    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44d8q\" (UniqueName: \"kubernetes.io/projected/b461004a-433c-4411-84df-b9aea9f682ab-kube-api-access-44d8q\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:23.086744 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.086698    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c8q8\" (UniqueName: \"kubernetes.io/projected/1c13ee40-c34c-4575-8865-aceeb71759d0-kube-api-access-2c8q8\") pod \"volume-data-source-validator-58c9f5f46d-cw5p4\" (UID: \"1c13ee40-c34c-4575-8865-aceeb71759d0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-cw5p4"
Nov 04 01:55:23.087181 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.086820    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:23.092153 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.089985    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9hwp\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-kube-api-access-k9hwp\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:23.092153 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.090715    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kslbj\" (UniqueName: \"kubernetes.io/projected/d09416ce-b095-4a28-9cb9-c0ad486ec03c-kube-api-access-kslbj\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-84j4p\" (UID: \"d09416ce-b095-4a28-9cb9-c0ad486ec03c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"
Nov 04 01:55:23.092153 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.091349    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b655g\" (UniqueName: \"kubernetes.io/projected/40363d65-7ece-4f05-8668-439a60582a18-kube-api-access-b655g\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:23.092153 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.091967    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8qmb\" (UniqueName: \"kubernetes.io/projected/99ef12a9-a676-4258-a3a8-e7fcd611d158-kube-api-access-k8qmb\") pod \"kas-connection-checker-685895c7c9-7gmmg\" (UID: \"99ef12a9-a676-4258-a3a8-e7fcd611d158\") " pod="kube-system/kas-connection-checker-685895c7c9-7gmmg"
Nov 04 01:55:23.122060 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.122029    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:23.135231 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.135205    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"
Nov 04 01:55:23.141536 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.141511    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-cw5p4"
Nov 04 01:55:23.153812 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.153780    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-p2kf6"
Nov 04 01:55:23.177449 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.177411    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5ljw9"
Nov 04 01:55:23.365195 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.365127    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-p48wv"
Nov 04 01:55:23.371447 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.371422    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"
Nov 04 01:55:23.378795 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.378765    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-7gmmg"
Nov 04 01:55:23.408722 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.408589    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-rzbgn"]
Nov 04 01:55:23.460923 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.460544    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"]
Nov 04 01:55:23.473223 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:55:23.473181    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod417744b4_5ed7_4b20_9960_7147d2d567ee.slice/crio-c7debcb3ff65b6f537be805308349656b4ce200749278be121b7d361958794b5 WatchSource:0}: Error finding container c7debcb3ff65b6f537be805308349656b4ce200749278be121b7d361958794b5: Status 404 returned error can't find the container with id c7debcb3ff65b6f537be805308349656b4ce200749278be121b7d361958794b5
Nov 04 01:55:23.518662 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.518632    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-2bzkj"]
Nov 04 01:55:23.520647 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.520473    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:23.520647 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.520521    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:23.520647 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.520560    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:23.520647 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.520629    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:55:23.521039 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.520696    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls podName:eddb65a3-0fa8-48b5-8ec9-4c939e189be4 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:24.520676991 +0000 UTC m=+33.242712447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-sr887" (UID: "eddb65a3-0fa8-48b5-8ec9-4c939e189be4") : secret "samples-operator-tls" not found
Nov 04 01:55:23.521039 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.520768    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:23.521039 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.520781    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8574f7698d-qmq69: secret "image-registry-tls" not found
Nov 04 01:55:23.521039 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.520816    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls podName:74e217e6-636d-4bab-b37f-c44dc0d8fe81 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:24.520804615 +0000 UTC m=+33.242840064 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls") pod "image-registry-8574f7698d-qmq69" (UID: "74e217e6-636d-4bab-b37f-c44dc0d8fe81") : secret "image-registry-tls" not found
Nov 04 01:55:23.521039 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.520996    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:55:23.521259 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.521065    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs podName:cc72c8e1-7e7c-4f53-946b-6ef94d6c5971 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:24.52104884 +0000 UTC m=+33.243084306 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs") pod "router-default-56bdbb678c-mvkmq" (UID: "cc72c8e1-7e7c-4f53-946b-6ef94d6c5971") : secret "router-metrics-certs-default" not found
Nov 04 01:55:23.521436 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.521409    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:23.521546 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.521471    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:23.521546 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:55:23.521476    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8ecf9a5_a6e5_460b_8cc9_44c996d57930.slice/crio-2b9d2187ceb34aa5ea3b605b593753a274826e1bd5adf4401fcdf9d83949f53b WatchSource:0}: Error finding container 2b9d2187ceb34aa5ea3b605b593753a274826e1bd5adf4401fcdf9d83949f53b: Status 404 returned error can't find the container with id 2b9d2187ceb34aa5ea3b605b593753a274826e1bd5adf4401fcdf9d83949f53b
Nov 04 01:55:23.521546 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.521500    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:23.521546 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.521538    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:23.521946 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.521590    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls podName:40363d65-7ece-4f05-8668-439a60582a18 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:24.521575816 +0000 UTC m=+33.243611272 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wzmdz" (UID: "40363d65-7ece-4f05-8668-439a60582a18") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:23.521946 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.521547    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-8rvcg\" (UID: \"c63f4e36-582d-43be-8575-3b79b7c4756d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:23.521946 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.521628    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:23.521946 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.521647    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle podName:cc72c8e1-7e7c-4f53-946b-6ef94d6c5971 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:24.521631598 +0000 UTC m=+33.243667045 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle") pod "router-default-56bdbb678c-mvkmq" (UID: "cc72c8e1-7e7c-4f53-946b-6ef94d6c5971") : configmap references non-existent config key: service-ca.crt
Nov 04 01:55:23.521946 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.521698    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:55:23.521946 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.521731    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls podName:b461004a-433c-4411-84df-b9aea9f682ab nodeName:}" failed. No retries permitted until 2026-11-04 01:55:24.521720472 +0000 UTC m=+33.243755930 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls") pod "dns-default-68kbx" (UID: "b461004a-433c-4411-84df-b9aea9f682ab") : secret "dns-default-metrics-tls" not found
Nov 04 01:55:23.521946 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.521772    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:55:23.521946 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.521779    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:23.521946 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.521796    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c59b47685-9jmnr: secret "image-registry-tls" not found
Nov 04 01:55:23.521946 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.521813    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert podName:c63f4e36-582d-43be-8575-3b79b7c4756d nodeName:}" failed. No retries permitted until 2026-11-04 01:55:24.521805791 +0000 UTC m=+33.243841233 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-8rvcg" (UID: "c63f4e36-582d-43be-8575-3b79b7c4756d") : secret "networking-console-plugin-cert" not found
Nov 04 01:55:23.521946 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:23.521838    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls podName:870962af-eac0-484d-9267-db8ab42ef13d nodeName:}" failed. No retries permitted until 2026-11-04 01:55:24.521824133 +0000 UTC m=+33.243859589 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls") pod "image-registry-6c59b47685-9jmnr" (UID: "870962af-eac0-484d-9267-db8ab42ef13d") : secret "image-registry-tls" not found
Nov 04 01:55:23.541769 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.541747    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-cw5p4"]
Nov 04 01:55:23.544855 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:55:23.544811    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c13ee40_c34c_4575_8865_aceeb71759d0.slice/crio-a05fdbadc57b9071e6a89a56a74230395c051348837b1d9588334d308a1d1132 WatchSource:0}: Error finding container a05fdbadc57b9071e6a89a56a74230395c051348837b1d9588334d308a1d1132: Status 404 returned error can't find the container with id a05fdbadc57b9071e6a89a56a74230395c051348837b1d9588334d308a1d1132
Nov 04 01:55:23.559620 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.548745    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"]
Nov 04 01:55:23.559620 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.549944    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-p2kf6"]
Nov 04 01:55:23.567599 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:55:23.567498    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d14f606_6852_4eff_b71f_5a8adcde5f84.slice/crio-435b4c22badc31787ef6c7a4bd26a6b89171efdeca310865c9bd0009b2c51208 WatchSource:0}: Error finding container 435b4c22badc31787ef6c7a4bd26a6b89171efdeca310865c9bd0009b2c51208: Status 404 returned error can't find the container with id 435b4c22badc31787ef6c7a4bd26a6b89171efdeca310865c9bd0009b2c51208
Nov 04 01:55:23.581498 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.581478    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5ljw9"]
Nov 04 01:55:23.582650 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:55:23.582626    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3db2db6_6f0f_4af7_9bb7_493df9107bb9.slice/crio-429e9d4fd1ba13c323733354917483fb7cac1ec84436db6173e645c1e8c6c66c WatchSource:0}: Error finding container 429e9d4fd1ba13c323733354917483fb7cac1ec84436db6173e645c1e8c6c66c: Status 404 returned error can't find the container with id 429e9d4fd1ba13c323733354917483fb7cac1ec84436db6173e645c1e8c6c66c
Nov 04 01:55:23.739708 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.739678    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:23.739845 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.739684    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:23.758558 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.758537    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-7gmmg"]
Nov 04 01:55:23.761502 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:55:23.761479    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99ef12a9_a676_4258_a3a8_e7fcd611d158.slice/crio-d6b0d38df1a9dc473b2721487dca1b06108f5eb26aee7a939748f3a0be7b4e86 WatchSource:0}: Error finding container d6b0d38df1a9dc473b2721487dca1b06108f5eb26aee7a939748f3a0be7b4e86: Status 404 returned error can't find the container with id d6b0d38df1a9dc473b2721487dca1b06108f5eb26aee7a939748f3a0be7b4e86
Nov 04 01:55:23.761662 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.761646    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-p48wv"]
Nov 04 01:55:23.767303 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.767266    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"]
Nov 04 01:55:23.769228 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:55:23.769201    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod698e630d_a5b0_43ce_9a12_aac9c60ab53c.slice/crio-0bb2fc810f7af835c9f906b6fe9baf7bdc2ec668b76db35ffe0481dc28faa034 WatchSource:0}: Error finding container 0bb2fc810f7af835c9f906b6fe9baf7bdc2ec668b76db35ffe0481dc28faa034: Status 404 returned error can't find the container with id 0bb2fc810f7af835c9f906b6fe9baf7bdc2ec668b76db35ffe0481dc28faa034
Nov 04 01:55:23.792974 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.792955    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 01:55:23.793065 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.793052    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-wpjt7\""
Nov 04 01:55:23.793120 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.793086    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tvbnh\""
Nov 04 01:55:23.793175 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.793144    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 01:55:23.938079 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.938054    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-7gmmg" event={"ID":"99ef12a9-a676-4258-a3a8-e7fcd611d158","Type":"ContainerStarted","Data":"d6b0d38df1a9dc473b2721487dca1b06108f5eb26aee7a939748f3a0be7b4e86"}
Nov 04 01:55:23.939522 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.939496    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-p2kf6" event={"ID":"4d14f606-6852-4eff-b71f-5a8adcde5f84","Type":"ContainerStarted","Data":"435b4c22badc31787ef6c7a4bd26a6b89171efdeca310865c9bd0009b2c51208"}
Nov 04 01:55:23.940583 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.940563    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t" event={"ID":"698e630d-a5b0-43ce-9a12-aac9c60ab53c","Type":"ContainerStarted","Data":"0bb2fc810f7af835c9f906b6fe9baf7bdc2ec668b76db35ffe0481dc28faa034"}
Nov 04 01:55:23.941850 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.941826    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-p48wv" event={"ID":"ef75aae9-bb5f-4e09-b374-fa398aadfae9","Type":"ContainerStarted","Data":"7b2f600b4ff99de30a4965ee52b741927d150a2474caddea46981c4634af4e12"}
Nov 04 01:55:23.943202 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.943181    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-cw5p4" event={"ID":"1c13ee40-c34c-4575-8865-aceeb71759d0","Type":"ContainerStarted","Data":"a05fdbadc57b9071e6a89a56a74230395c051348837b1d9588334d308a1d1132"}
Nov 04 01:55:23.944616 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.944565    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5ljw9" event={"ID":"f3db2db6-6f0f-4af7-9bb7-493df9107bb9","Type":"ContainerStarted","Data":"429e9d4fd1ba13c323733354917483fb7cac1ec84436db6173e645c1e8c6c66c"}
Nov 04 01:55:23.945946 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.945925    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p" event={"ID":"d09416ce-b095-4a28-9cb9-c0ad486ec03c","Type":"ContainerStarted","Data":"5ba80f3ee290ca94c7acaf25bd3155ad37d11616addf83d241287ef2a7259bfa"}
Nov 04 01:55:23.947154 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.947128    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-2bzkj" event={"ID":"a8ecf9a5-a6e5-460b-8cc9-44c996d57930","Type":"ContainerStarted","Data":"2b9d2187ceb34aa5ea3b605b593753a274826e1bd5adf4401fcdf9d83949f53b"}
Nov 04 01:55:23.948234 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.948209    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg" event={"ID":"417744b4-5ed7-4b20-9960-7147d2d567ee","Type":"ContainerStarted","Data":"c7debcb3ff65b6f537be805308349656b4ce200749278be121b7d361958794b5"}
Nov 04 01:55:23.949721 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.949697    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-rzbgn" event={"ID":"c07ce523-d075-4203-be44-3f7d5af7e0d7","Type":"ContainerStarted","Data":"3eb4da23aa3008e4e181d332367ae9370db6666e369362d97fad82bff442b002"}
Nov 04 01:55:23.949815 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:23.949737    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-rzbgn" event={"ID":"c07ce523-d075-4203-be44-3f7d5af7e0d7","Type":"ContainerStarted","Data":"f51632b5c6188bd247ed22b62c42f49a22426c075ccc595088ba3744454f5aa8"}
Nov 04 01:55:24.532433 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:24.532398    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:24.532433 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:24.532437    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:24.532628 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:24.532468    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:24.532628 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:24.532500    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:24.532628 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:24.532569    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:24.532628 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:24.532603    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:24.532765 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:24.532651    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-8rvcg\" (UID: \"c63f4e36-582d-43be-8575-3b79b7c4756d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:24.532765 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:24.532680    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:24.532928 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:24.532911    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:55:24.532984 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:24.532975    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls podName:b461004a-433c-4411-84df-b9aea9f682ab nodeName:}" failed. No retries permitted until 2026-11-04 01:55:26.532957681 +0000 UTC m=+35.254993124 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls") pod "dns-default-68kbx" (UID: "b461004a-433c-4411-84df-b9aea9f682ab") : secret "dns-default-metrics-tls" not found
Nov 04 01:55:24.533051 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:24.533040    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:55:24.533123 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:24.533115    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls podName:eddb65a3-0fa8-48b5-8ec9-4c939e189be4 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:26.53306485 +0000 UTC m=+35.255100289 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-sr887" (UID: "eddb65a3-0fa8-48b5-8ec9-4c939e189be4") : secret "samples-operator-tls" not found
Nov 04 01:55:24.533173 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:24.533163    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:24.533204 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:24.533175    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8574f7698d-qmq69: secret "image-registry-tls" not found
Nov 04 01:55:24.533204 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:24.533196    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls podName:74e217e6-636d-4bab-b37f-c44dc0d8fe81 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:26.533190124 +0000 UTC m=+35.255225563 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls") pod "image-registry-8574f7698d-qmq69" (UID: "74e217e6-636d-4bab-b37f-c44dc0d8fe81") : secret "image-registry-tls" not found
Nov 04 01:55:24.533311 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:24.533232    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:55:24.533311 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:24.533249    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs podName:cc72c8e1-7e7c-4f53-946b-6ef94d6c5971 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:26.533243816 +0000 UTC m=+35.255279254 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs") pod "router-default-56bdbb678c-mvkmq" (UID: "cc72c8e1-7e7c-4f53-946b-6ef94d6c5971") : secret "router-metrics-certs-default" not found
Nov 04 01:55:24.533311 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:24.533299    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:24.533462 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:24.533378    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:24.533462 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:24.533392    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c59b47685-9jmnr: secret "image-registry-tls" not found
Nov 04 01:55:24.533462 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:24.533383    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls podName:40363d65-7ece-4f05-8668-439a60582a18 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:26.533320847 +0000 UTC m=+35.255356291 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wzmdz" (UID: "40363d65-7ece-4f05-8668-439a60582a18") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:24.533462 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:24.533421    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:55:24.533462 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:24.533432    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls podName:870962af-eac0-484d-9267-db8ab42ef13d nodeName:}" failed. No retries permitted until 2026-11-04 01:55:26.533423999 +0000 UTC m=+35.255459441 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls") pod "image-registry-6c59b47685-9jmnr" (UID: "870962af-eac0-484d-9267-db8ab42ef13d") : secret "image-registry-tls" not found
Nov 04 01:55:24.533710 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:24.533489    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle podName:cc72c8e1-7e7c-4f53-946b-6ef94d6c5971 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:26.533470414 +0000 UTC m=+35.255505859 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle") pod "router-default-56bdbb678c-mvkmq" (UID: "cc72c8e1-7e7c-4f53-946b-6ef94d6c5971") : configmap references non-existent config key: service-ca.crt
Nov 04 01:55:24.533710 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:24.533518    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert podName:c63f4e36-582d-43be-8575-3b79b7c4756d nodeName:}" failed. No retries permitted until 2026-11-04 01:55:26.533497301 +0000 UTC m=+35.255532740 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-8rvcg" (UID: "c63f4e36-582d-43be-8575-3b79b7c4756d") : secret "networking-console-plugin-cert" not found
Nov 04 01:55:24.928415 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:24.928275    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-rzbgn" podStartSLOduration=381.928256023 podStartE2EDuration="6m21.928256023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:49:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:24.927828225 +0000 UTC m=+33.649863711" watchObservedRunningTime="2026-11-04 01:55:24.928256023 +0000 UTC m=+33.650291484"
Nov 04 01:55:24.956983 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:24.956952    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-7gmmg" event={"ID":"99ef12a9-a676-4258-a3a8-e7fcd611d158","Type":"ContainerStarted","Data":"72127009a2fc495a7502c30438df1cdc9a2d867e4dd16a066345c223ff4508fc"}
Nov 04 01:55:24.962774 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:24.962711    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-p48wv" event={"ID":"ef75aae9-bb5f-4e09-b374-fa398aadfae9","Type":"ContainerStarted","Data":"345b0ea9358c6071bf074112f34ca6ade8b1811caef23d60d62ced0801cc0479"}
Nov 04 01:55:25.077065 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:25.075692    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-7gmmg" podStartSLOduration=382.075673671 podStartE2EDuration="6m22.075673671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:49:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:25.07476467 +0000 UTC m=+33.796800159" watchObservedRunningTime="2026-11-04 01:55:25.075673671 +0000 UTC m=+33.797709133"
Nov 04 01:55:25.198513 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:25.198455    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-p48wv" podStartSLOduration=382.198435947 podStartE2EDuration="6m22.198435947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:49:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:25.197140903 +0000 UTC m=+33.919176389" watchObservedRunningTime="2026-11-04 01:55:25.198435947 +0000 UTC m=+33.920471414"
Nov 04 01:55:25.543156 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:25.543060    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:25.548896 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:25.548838    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:25.613409 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:25.613377    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:25.644753 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:25.644229    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs\") pod \"network-metrics-daemon-74j82\" (UID: \"1536e272-39b6-4702-a35d-0130eef3e290\") " pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:25.644753 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:25.644312    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56z2g\" (UniqueName: \"kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g\") pod \"network-check-target-g6xmn\" (UID: \"213808a8-b7ec-4fc6-845b-d2648393d5bc\") " pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:25.644753 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:25.644413    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 01:55:25.644753 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:25.644499    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs podName:1536e272-39b6-4702-a35d-0130eef3e290 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:57.644479425 +0000 UTC m=+66.366514881 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs") pod "network-metrics-daemon-74j82" (UID: "1536e272-39b6-4702-a35d-0130eef3e290") : secret "metrics-daemon-secret" not found
Nov 04 01:55:25.651867 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:25.651652    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56z2g\" (UniqueName: \"kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g\") pod \"network-check-target-g6xmn\" (UID: \"213808a8-b7ec-4fc6-845b-d2648393d5bc\") " pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:25.750406 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:25.750372    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:26.046135 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:26.046037    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x48xl"]
Nov 04 01:55:26.249647 ip-10-0-134-204 kubenswrapper[2574]: W1104 01:55:26.249599    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod213808a8_b7ec_4fc6_845b_d2648393d5bc.slice/crio-0aef766ce944d7a0701fee5cc54fb4951ca982cdcdcedf2da8776b68d9f58b07 WatchSource:0}: Error finding container 0aef766ce944d7a0701fee5cc54fb4951ca982cdcdcedf2da8776b68d9f58b07: Status 404 returned error can't find the container with id 0aef766ce944d7a0701fee5cc54fb4951ca982cdcdcedf2da8776b68d9f58b07
Nov 04 01:55:26.249938 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:26.249913    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g6xmn"]
Nov 04 01:55:26.553967 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:26.553869    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:26.553967 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:26.553937    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:26.554186 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:26.553984    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:26.554186 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:26.554016    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:26.554186 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:26.554048    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:55:26.554186 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:26.554120    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs podName:cc72c8e1-7e7c-4f53-946b-6ef94d6c5971 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:30.554103656 +0000 UTC m=+39.276139121 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs") pod "router-default-56bdbb678c-mvkmq" (UID: "cc72c8e1-7e7c-4f53-946b-6ef94d6c5971") : secret "router-metrics-certs-default" not found
Nov 04 01:55:26.554186 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:26.554152    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:55:26.554403 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:26.554204    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert podName:c63f4e36-582d-43be-8575-3b79b7c4756d nodeName:}" failed. No retries permitted until 2026-11-04 01:55:30.554189247 +0000 UTC m=+39.276224701 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-8rvcg" (UID: "c63f4e36-582d-43be-8575-3b79b7c4756d") : secret "networking-console-plugin-cert" not found
Nov 04 01:55:26.554403 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:26.554266    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:26.554403 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:26.554316    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls podName:40363d65-7ece-4f05-8668-439a60582a18 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:30.554305227 +0000 UTC m=+39.276340691 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wzmdz" (UID: "40363d65-7ece-4f05-8668-439a60582a18") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:26.554403 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:26.554377    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle podName:cc72c8e1-7e7c-4f53-946b-6ef94d6c5971 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:30.554369439 +0000 UTC m=+39.276404917 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle") pod "router-default-56bdbb678c-mvkmq" (UID: "cc72c8e1-7e7c-4f53-946b-6ef94d6c5971") : configmap references non-existent config key: service-ca.crt
Nov 04 01:55:26.554546 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:26.554059    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-8rvcg\" (UID: \"c63f4e36-582d-43be-8575-3b79b7c4756d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:26.554546 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:26.554434    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:26.554546 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:26.554443    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c59b47685-9jmnr: secret "image-registry-tls" not found
Nov 04 01:55:26.554546 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:26.554454    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:26.554546 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:26.554462    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls podName:870962af-eac0-484d-9267-db8ab42ef13d nodeName:}" failed. No retries permitted until 2026-11-04 01:55:30.554456484 +0000 UTC m=+39.276491982 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls") pod "image-registry-6c59b47685-9jmnr" (UID: "870962af-eac0-484d-9267-db8ab42ef13d") : secret "image-registry-tls" not found
Nov 04 01:55:26.554546 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:26.554505    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:55:26.554546 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:26.554527    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls podName:b461004a-433c-4411-84df-b9aea9f682ab nodeName:}" failed. No retries permitted until 2026-11-04 01:55:30.554520281 +0000 UTC m=+39.276555761 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls") pod "dns-default-68kbx" (UID: "b461004a-433c-4411-84df-b9aea9f682ab") : secret "dns-default-metrics-tls" not found
Nov 04 01:55:26.554546 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:26.554540    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:26.554940 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:26.554563    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:26.554940 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:26.554645    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:26.554940 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:26.554655    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8574f7698d-qmq69: secret "image-registry-tls" not found
Nov 04 01:55:26.554940 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:26.554689    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls podName:74e217e6-636d-4bab-b37f-c44dc0d8fe81 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:30.554678961 +0000 UTC m=+39.276714405 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls") pod "image-registry-8574f7698d-qmq69" (UID: "74e217e6-636d-4bab-b37f-c44dc0d8fe81") : secret "image-registry-tls" not found
Nov 04 01:55:26.554940 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:26.554718    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:55:26.554940 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:26.554780    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls podName:eddb65a3-0fa8-48b5-8ec9-4c939e189be4 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:30.554764654 +0000 UTC m=+39.276800111 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-sr887" (UID: "eddb65a3-0fa8-48b5-8ec9-4c939e189be4") : secret "samples-operator-tls" not found
Nov 04 01:55:26.971522 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:26.971485    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g6xmn" event={"ID":"213808a8-b7ec-4fc6-845b-d2648393d5bc","Type":"ContainerStarted","Data":"0aef766ce944d7a0701fee5cc54fb4951ca982cdcdcedf2da8776b68d9f58b07"}
Nov 04 01:55:26.975947 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:26.975154    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-cw5p4" event={"ID":"1c13ee40-c34c-4575-8865-aceeb71759d0","Type":"ContainerStarted","Data":"c5264688ed66c640ce74dacb123dabb3862c7c9766f6baeea13a0734aa1d3038"}
Nov 04 01:55:26.977643 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:26.977581    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x48xl" event={"ID":"a994b38f-0096-4fff-b49c-e6d99dc5a6be","Type":"ContainerStarted","Data":"39664ccb3f9a6204ddffa1cc3448663e1c16d4aaf62db97d86b97f6ca54b5bb0"}
Nov 04 01:55:27.094427 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:27.094365    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-cw5p4" podStartSLOduration=351.778337997 podStartE2EDuration="5m54.094346106s" totalImagesPullingTime="2.316008109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:49:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:27.092869836 +0000 UTC m=+35.814905299" watchObservedRunningTime="2026-11-04 01:55:27.094346106 +0000 UTC m=+35.816381569"
Nov 04 01:55:28.987144 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:28.986708    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-2bzkj" event={"ID":"a8ecf9a5-a6e5-460b-8cc9-44c996d57930","Type":"ContainerStarted","Data":"c5729e59dae23db2ab5e4e64cccd538f9b4f94a917a6697f37026a8be5b20e7d"}
Nov 04 01:55:28.990027 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:28.990003    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/0.log"
Nov 04 01:55:28.990164 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:28.990041    2574 generic.go:356] "Generic (PLEG): container finished" podID="417744b4-5ed7-4b20-9960-7147d2d567ee" containerID="30da12c4e7b23c164f0e4071b106fdbad78e4f8276ec4ce9643b3f03b03876c4" exitCode=255
Nov 04 01:55:28.990164 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:28.990088    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg" event={"ID":"417744b4-5ed7-4b20-9960-7147d2d567ee","Type":"ContainerDied","Data":"30da12c4e7b23c164f0e4071b106fdbad78e4f8276ec4ce9643b3f03b03876c4"}
Nov 04 01:55:28.990426 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:28.990408    2574 scope.go:122] "RemoveContainer" containerID="30da12c4e7b23c164f0e4071b106fdbad78e4f8276ec4ce9643b3f03b03876c4"
Nov 04 01:55:29.107676 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.106688    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-2bzkj" podStartSLOduration=341.406249494 podStartE2EDuration="5m46.106675098s" totalImagesPullingTime="4.700425604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:49:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:29.106210286 +0000 UTC m=+37.828245758" watchObservedRunningTime="2026-11-04 01:55:29.106675098 +0000 UTC m=+37.828710590"
Nov 04 01:55:29.650877 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.650843    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fkhvv"]
Nov 04 01:55:29.675337 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.675276    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:29.714844 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.714811    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fkhvv"]
Nov 04 01:55:29.796790 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.796711    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/789e64a1-501d-46d9-a58d-14f786410301-data-volume\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:29.797263 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.796950    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/789e64a1-501d-46d9-a58d-14f786410301-crio-socket\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:29.797263 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.797040    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/789e64a1-501d-46d9-a58d-14f786410301-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:29.797263 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.797082    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78j5f\" (UniqueName: \"kubernetes.io/projected/789e64a1-501d-46d9-a58d-14f786410301-kube-api-access-78j5f\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:29.797263 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.797119    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:29.823346 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.814534    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 01:55:29.823346 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.814562    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 01:55:29.823346 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.814659    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mj9m5\""
Nov 04 01:55:29.898157 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.898116    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/789e64a1-501d-46d9-a58d-14f786410301-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:29.898378 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.898186    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78j5f\" (UniqueName: \"kubernetes.io/projected/789e64a1-501d-46d9-a58d-14f786410301-kube-api-access-78j5f\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:29.898378 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.898222    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:29.898378 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.898247    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/789e64a1-501d-46d9-a58d-14f786410301-data-volume\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:29.898526 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.898411    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/789e64a1-501d-46d9-a58d-14f786410301-crio-socket\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:29.898573 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.898558    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/789e64a1-501d-46d9-a58d-14f786410301-crio-socket\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:29.898676 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:29.898656    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:55:29.898753 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:29.898740    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/789e64a1-501d-46d9-a58d-14f786410301-kube-rbac-proxy-cm podName:789e64a1-501d-46d9-a58d-14f786410301 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:30.398707907 +0000 UTC m=+39.120743363 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/789e64a1-501d-46d9-a58d-14f786410301-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fkhvv" (UID: "789e64a1-501d-46d9-a58d-14f786410301") : configmap "kube-rbac-proxy" not found
Nov 04 01:55:29.899242 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:29.899217    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:29.899370 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:29.899270    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls podName:789e64a1-501d-46d9-a58d-14f786410301 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:30.399255746 +0000 UTC m=+39.121291186 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkhvv" (UID: "789e64a1-501d-46d9-a58d-14f786410301") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:29.899603 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.899580    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/789e64a1-501d-46d9-a58d-14f786410301-data-volume\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:29.983033 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.983000    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78j5f\" (UniqueName: \"kubernetes.io/projected/789e64a1-501d-46d9-a58d-14f786410301-kube-api-access-78j5f\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:29.999722 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:29.999630    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/1.log"
Nov 04 01:55:30.000804 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.000554    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/0.log"
Nov 04 01:55:30.000804 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.000594    2574 generic.go:356] "Generic (PLEG): container finished" podID="417744b4-5ed7-4b20-9960-7147d2d567ee" containerID="5a240684c33f12cf1f818a2ddee27b424185731bb59eb98661d7f29c71edadc0" exitCode=255
Nov 04 01:55:30.000804 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.000712    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg" event={"ID":"417744b4-5ed7-4b20-9960-7147d2d567ee","Type":"ContainerDied","Data":"5a240684c33f12cf1f818a2ddee27b424185731bb59eb98661d7f29c71edadc0"}
Nov 04 01:55:30.001952 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.001048    2574 scope.go:122] "RemoveContainer" containerID="30da12c4e7b23c164f0e4071b106fdbad78e4f8276ec4ce9643b3f03b03876c4"
Nov 04 01:55:30.001952 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.001515    2574 scope.go:122] "RemoveContainer" containerID="5a240684c33f12cf1f818a2ddee27b424185731bb59eb98661d7f29c71edadc0"
Nov 04 01:55:30.001952 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.001729    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-jlqmg_openshift-console-operator(417744b4-5ed7-4b20-9960-7147d2d567ee)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg" podUID="417744b4-5ed7-4b20-9960-7147d2d567ee"
Nov 04 01:55:30.006606 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.006071    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t" event={"ID":"698e630d-a5b0-43ce-9a12-aac9c60ab53c","Type":"ContainerStarted","Data":"9d66a760b2ca4602a40f00ed106ea89ea3fb426ed38d607fa866e07a23969d55"}
Nov 04 01:55:30.010009 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.009987    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p" event={"ID":"d09416ce-b095-4a28-9cb9-c0ad486ec03c","Type":"ContainerStarted","Data":"a4442abda16fbcd14764c8106809dac533129cd017ab86de08ac4aed3c790f97"}
Nov 04 01:55:30.257411 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.257345    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t" podStartSLOduration=346.532951091 podStartE2EDuration="5m52.257329106s" totalImagesPullingTime="5.724378015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:30.256838271 +0000 UTC m=+38.978873735" watchObservedRunningTime="2026-11-04 01:55:30.257329106 +0000 UTC m=+38.979364569"
Nov 04 01:55:30.377524 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.377469    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p" podStartSLOduration=341.444020704 podStartE2EDuration="5m47.37745329s" totalImagesPullingTime="5.933432586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:49:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:30.37599549 +0000 UTC m=+39.098030950" watchObservedRunningTime="2026-11-04 01:55:30.37745329 +0000 UTC m=+39.099488750"
Nov 04 01:55:30.403659 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.403621    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/789e64a1-501d-46d9-a58d-14f786410301-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:30.403801 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.403700    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:30.404006 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.403987    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:30.404077 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.404052    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls podName:789e64a1-501d-46d9-a58d-14f786410301 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:31.404033258 +0000 UTC m=+40.126068709 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkhvv" (UID: "789e64a1-501d-46d9-a58d-14f786410301") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:30.404140 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.404092    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:55:30.404140 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.404124    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/789e64a1-501d-46d9-a58d-14f786410301-kube-rbac-proxy-cm podName:789e64a1-501d-46d9-a58d-14f786410301 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:31.404112902 +0000 UTC m=+40.126148341 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/789e64a1-501d-46d9-a58d-14f786410301-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fkhvv" (UID: "789e64a1-501d-46d9-a58d-14f786410301") : configmap "kube-rbac-proxy" not found
Nov 04 01:55:30.605994 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.605909    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:30.605994 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.605975    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:30.606233 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.606030    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:30.606233 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.606080    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:30.606233 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.606105    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8574f7698d-qmq69: secret "image-registry-tls" not found
Nov 04 01:55:30.606233 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.606166    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls podName:74e217e6-636d-4bab-b37f-c44dc0d8fe81 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.606146771 +0000 UTC m=+47.328182227 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls") pod "image-registry-8574f7698d-qmq69" (UID: "74e217e6-636d-4bab-b37f-c44dc0d8fe81") : secret "image-registry-tls" not found
Nov 04 01:55:30.606233 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.606087    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:30.606233 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.606190    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle podName:cc72c8e1-7e7c-4f53-946b-6ef94d6c5971 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.606174799 +0000 UTC m=+47.328210245 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle") pod "router-default-56bdbb678c-mvkmq" (UID: "cc72c8e1-7e7c-4f53-946b-6ef94d6c5971") : configmap references non-existent config key: service-ca.crt
Nov 04 01:55:30.606233 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.606220    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:30.606636 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.606191    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:55:30.606636 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.606277    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls podName:40363d65-7ece-4f05-8668-439a60582a18 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.606260122 +0000 UTC m=+47.328295575 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wzmdz" (UID: "40363d65-7ece-4f05-8668-439a60582a18") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:30.606636 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.606307    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:30.606636 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.606320    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c59b47685-9jmnr: secret "image-registry-tls" not found
Nov 04 01:55:30.606636 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.606334    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs podName:cc72c8e1-7e7c-4f53-946b-6ef94d6c5971 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.606319827 +0000 UTC m=+47.328355289 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs") pod "router-default-56bdbb678c-mvkmq" (UID: "cc72c8e1-7e7c-4f53-946b-6ef94d6c5971") : secret "router-metrics-certs-default" not found
Nov 04 01:55:30.606636 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.606354    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls podName:870962af-eac0-484d-9267-db8ab42ef13d nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.606341774 +0000 UTC m=+47.328377244 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls") pod "image-registry-6c59b47685-9jmnr" (UID: "870962af-eac0-484d-9267-db8ab42ef13d") : secret "image-registry-tls" not found
Nov 04 01:55:30.606636 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.606224    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:30.606636 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.606405    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-8rvcg\" (UID: \"c63f4e36-582d-43be-8575-3b79b7c4756d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:30.606636 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.606433    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:30.606636 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.606544    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:55:30.606636 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.606558    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:55:30.606636 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.606575    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls podName:b461004a-433c-4411-84df-b9aea9f682ab nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.606565557 +0000 UTC m=+47.328601010 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls") pod "dns-default-68kbx" (UID: "b461004a-433c-4411-84df-b9aea9f682ab") : secret "dns-default-metrics-tls" not found
Nov 04 01:55:30.606636 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:30.606606    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:30.606636 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.606642    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:55:30.607304 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.606674    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls podName:eddb65a3-0fa8-48b5-8ec9-4c939e189be4 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.606663036 +0000 UTC m=+47.328698489 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-sr887" (UID: "eddb65a3-0fa8-48b5-8ec9-4c939e189be4") : secret "samples-operator-tls" not found
Nov 04 01:55:30.607304 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:30.606691    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert podName:c63f4e36-582d-43be-8575-3b79b7c4756d nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.606679274 +0000 UTC m=+47.328714726 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-8rvcg" (UID: "c63f4e36-582d-43be-8575-3b79b7c4756d") : secret "networking-console-plugin-cert" not found
Nov 04 01:55:31.014182 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:31.014154    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/1.log"
Nov 04 01:55:31.014628 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:31.014560    2574 scope.go:122] "RemoveContainer" containerID="5a240684c33f12cf1f818a2ddee27b424185731bb59eb98661d7f29c71edadc0"
Nov 04 01:55:31.014829 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:31.014793    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-jlqmg_openshift-console-operator(417744b4-5ed7-4b20-9960-7147d2d567ee)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg" podUID="417744b4-5ed7-4b20-9960-7147d2d567ee"
Nov 04 01:55:31.016265 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:31.016238    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-p2kf6" event={"ID":"4d14f606-6852-4eff-b71f-5a8adcde5f84","Type":"ContainerStarted","Data":"1c1d2a49dc22133f5ba3d22cc76f0464894d809c52a6f52a6ff4f35072214e0a"}
Nov 04 01:55:31.250502 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:31.250436    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-p2kf6" podStartSLOduration=341.518972027 podStartE2EDuration="5m48.250414631s" totalImagesPullingTime="6.731442604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:49:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:31.249299657 +0000 UTC m=+39.971335112" watchObservedRunningTime="2026-11-04 01:55:31.250414631 +0000 UTC m=+39.972450096"
Nov 04 01:55:31.415003 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:31.414973    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/789e64a1-501d-46d9-a58d-14f786410301-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:31.415112 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:31.415019    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:31.415160 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:31.415099    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:55:31.415160 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:31.415120    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:31.415221 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:31.415170    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/789e64a1-501d-46d9-a58d-14f786410301-kube-rbac-proxy-cm podName:789e64a1-501d-46d9-a58d-14f786410301 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:33.415154651 +0000 UTC m=+42.137190103 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/789e64a1-501d-46d9-a58d-14f786410301-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fkhvv" (UID: "789e64a1-501d-46d9-a58d-14f786410301") : configmap "kube-rbac-proxy" not found
Nov 04 01:55:31.415221 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:31.415186    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls podName:789e64a1-501d-46d9-a58d-14f786410301 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:33.415179725 +0000 UTC m=+42.137215210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkhvv" (UID: "789e64a1-501d-46d9-a58d-14f786410301") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:32.021520 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:32.021482    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5ljw9" event={"ID":"f3db2db6-6f0f-4af7-9bb7-493df9107bb9","Type":"ContainerStarted","Data":"626232d0e3733a5b4c314d9dac01b50b70cfd40a1ccd8991707fedca87f788b6"}
Nov 04 01:55:32.023018 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:32.022994    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g6xmn" event={"ID":"213808a8-b7ec-4fc6-845b-d2648393d5bc","Type":"ContainerStarted","Data":"3185146373bec73cc2c38babc34ba3af3652587bb2a3beabd4098cd13bcf16be"}
Nov 04 01:55:32.138869 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:32.138816    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5ljw9" podStartSLOduration=2.35122402 podStartE2EDuration="10.138801852s" totalImagesPullingTime="7.787577832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:55:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:32.138256829 +0000 UTC m=+40.860292289" watchObservedRunningTime="2026-11-04 01:55:32.138801852 +0000 UTC m=+40.860837312"
Nov 04 01:55:32.255553 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:32.255501    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-g6xmn" podStartSLOduration=35.126360344 podStartE2EDuration="40.255484012s" totalImagesPullingTime="5.129123668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:54:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:32.254380745 +0000 UTC m=+40.976416206" watchObservedRunningTime="2026-11-04 01:55:32.255484012 +0000 UTC m=+40.977519475"
Nov 04 01:55:33.026445 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:33.026370    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:33.088063 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:33.088024    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:33.088344 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:33.088095    2574 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:33.088538 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:33.088522    2574 scope.go:122] "RemoveContainer" containerID="5a240684c33f12cf1f818a2ddee27b424185731bb59eb98661d7f29c71edadc0"
Nov 04 01:55:33.088749 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:33.088727    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-jlqmg_openshift-console-operator(417744b4-5ed7-4b20-9960-7147d2d567ee)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg" podUID="417744b4-5ed7-4b20-9960-7147d2d567ee"
Nov 04 01:55:33.435790 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:33.435707    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/789e64a1-501d-46d9-a58d-14f786410301-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:33.435790 ip-10-0-134-204 kubenswrapper[2574]: I1104 01:55:33.435771    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:33.435996 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:33.435877    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:33.435996 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:33.435896    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:55:33.435996 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:33.435951    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/789e64a1-501d-46d9-a58d-14f786410301-kube-rbac-proxy-cm podName:789e64a1-501d-46d9-a58d-14f786410301 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:37.435937846 +0000 UTC m=+46.157973298 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/789e64a1-501d-46d9-a58d-14f786410301-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fkhvv" (UID: "789e64a1-501d-46d9-a58d-14f786410301") : configmap "kube-rbac-proxy" not found
Nov 04 01:55:33.435996 ip-10-0-134-204 kubenswrapper[2574]: E1104 01:55:33.435969    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls podName:789e64a1-501d-46d9-a58d-14f786410301 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:37.435959583 +0000 UTC m=+46.157995029 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkhvv" (UID: "789e64a1-501d-46d9-a58d-14f786410301") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:36.718159 ip-10-0-134-204 systemd[1]: Stopping Kubernetes Kubelet...
Nov 04 01:55:36.727659 ip-10-0-134-204 systemd[1]: kubelet.service: Deactivated successfully.
Nov 04 01:55:36.727827 ip-10-0-134-204 systemd[1]: Stopped Kubernetes Kubelet.
Nov 04 01:55:36.727879 ip-10-0-134-204 systemd[1]: kubelet.service: Consumed 3.071s CPU time.
Nov 04 01:55:36.742047 ip-10-0-134-204 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 01:55:36.813962 ip-10-0-134-204 kubenswrapper[4865]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:55:36.813962 ip-10-0-134-204 kubenswrapper[4865]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 01:55:36.813962 ip-10-0-134-204 kubenswrapper[4865]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:55:36.813962 ip-10-0-134-204 kubenswrapper[4865]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:55:36.816030 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.815941    4865 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:55:36.816030 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816030    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816034    4865 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816037    4865 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816041    4865 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816045    4865 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816048    4865 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816051    4865 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816054    4865 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816056    4865 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816059    4865 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816062    4865 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816064    4865 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816067    4865 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816070    4865 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816072    4865 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816075    4865 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816077    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816080    4865 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816088    4865 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:55:36.816099 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816093    4865 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816101    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816105    4865 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816108    4865 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816111    4865 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816114    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816117    4865 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816120    4865 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816122    4865 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816126    4865 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816130    4865 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816133    4865 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816136    4865 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816139    4865 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816143    4865 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816146    4865 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816148    4865 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816151    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816153    4865 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816157    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:55:36.816594 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816159    4865 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816162    4865 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816165    4865 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816167    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816170    4865 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816172    4865 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816175    4865 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816177    4865 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816180    4865 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816182    4865 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816185    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816187    4865 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816191    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816193    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816197    4865 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816200    4865 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816203    4865 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816205    4865 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816208    4865 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816210    4865 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:55:36.817088 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816213    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816216    4865 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816219    4865 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816222    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816224    4865 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816227    4865 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816229    4865 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816232    4865 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816235    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816238    4865 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816241    4865 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816243    4865 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816247    4865 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816249    4865 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816252    4865 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816255    4865 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816257    4865 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816260    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816263    4865 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:55:36.817630 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816265    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816268    4865 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816271    4865 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816273    4865 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816276    4865 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816291    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816295    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816300    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816302    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816305    4865 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816308    4865 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816311    4865 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816313    4865 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816316    4865 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816319    4865 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816321    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816324    4865 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816327    4865 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816329    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816332    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:55:36.818117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816335    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816337    4865 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816340    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816343    4865 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816345    4865 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816350    4865 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.816353    4865 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817112    4865 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817118    4865 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817121    4865 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817125    4865 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817127    4865 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817130    4865 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817133    4865 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817135    4865 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817138    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817141    4865 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817144    4865 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817146    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817150    4865 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:55:36.818635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817153    4865 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817156    4865 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817159    4865 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817161    4865 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817164    4865 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817166    4865 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817169    4865 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817172    4865 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817174    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817177    4865 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817179    4865 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817182    4865 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817184    4865 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817187    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817189    4865 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817192    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817194    4865 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817197    4865 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817199    4865 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817203    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:55:36.819166 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817205    4865 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817210    4865 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817215    4865 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817219    4865 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817222    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817225    4865 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817227    4865 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817230    4865 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817233    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817236    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817238    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817241    4865 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817244    4865 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817246    4865 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817249    4865 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817251    4865 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817254    4865 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817257    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817259    4865 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:55:36.819677 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817262    4865 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817264    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817267    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817270    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817273    4865 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817276    4865 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817294    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817299    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817302    4865 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817305    4865 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817308    4865 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817310    4865 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817313    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817317    4865 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817320    4865 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817324    4865 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817327    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817330    4865 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817332    4865 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:55:36.820313 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817335    4865 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817338    4865 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817341    4865 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817343    4865 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817346    4865 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817349    4865 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817352    4865 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817355    4865 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817358    4865 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817360    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817363    4865 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817366    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817368    4865 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817371    4865 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817373    4865 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817376    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817378    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817381    4865 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817385    4865 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:55:36.820846 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817388    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817391    4865 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817393    4865 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817396    4865 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817399    4865 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817402    4865 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817404    4865 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817407    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817411    4865 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817413    4865 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817416    4865 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817419    4865 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817421    4865 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817424    4865 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817426    4865 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817429    4865 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817946    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817953    4865 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817956    4865 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817959    4865 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:55:36.821354 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817962    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817965    4865 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817968    4865 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817971    4865 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817974    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817976    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817979    4865 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817982    4865 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817986    4865 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817990    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817994    4865 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817996    4865 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.817999    4865 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818002    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818004    4865 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818007    4865 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818012    4865 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818015    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:55:36.821912 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818017    4865 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818020    4865 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818023    4865 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818026    4865 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818028    4865 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818031    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818034    4865 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818036    4865 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818039    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818041    4865 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818044    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818047    4865 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818049    4865 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818052    4865 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818054    4865 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818057    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818059    4865 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818062    4865 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818065    4865 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818067    4865 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:55:36.822389 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818070    4865 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818073    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818076    4865 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818078    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818081    4865 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818084    4865 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818086    4865 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818089    4865 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818091    4865 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818093    4865 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818099    4865 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818103    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818106    4865 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818110    4865 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818113    4865 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818117    4865 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818120    4865 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818123    4865 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818126    4865 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:55:36.822897 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818129    4865 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818132    4865 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818135    4865 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818139    4865 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818142    4865 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818144    4865 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818147    4865 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818150    4865 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818152    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818155    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818158    4865 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818160    4865 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818163    4865 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818166    4865 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818169    4865 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818171    4865 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818174    4865 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818176    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818179    4865 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818183    4865 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:55:36.823387 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818185    4865 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818188    4865 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818190    4865 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818194    4865 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818197    4865 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818200    4865 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818202    4865 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818205    4865 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818207    4865 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818210    4865 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818213    4865 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818215    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818218    4865 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818220    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818223    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818225    4865 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818228    4865 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818230    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818232    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818235    4865 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:55:36.823883 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818237    4865 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818240    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818242    4865 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818245    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.818247    4865 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818363    4865 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818372    4865 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818383    4865 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818387    4865 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818392    4865 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818395    4865 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818399    4865 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818403    4865 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818406    4865 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818409    4865 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818415    4865 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818418    4865 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818421    4865 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818425    4865 flags.go:64] FLAG: --cgroup-root=""
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818427    4865 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818430    4865 flags.go:64] FLAG: --client-ca-file=""
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818433    4865 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818437    4865 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818441    4865 flags.go:64] FLAG: --cluster-domain=""
Nov 04 01:55:36.824513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818444    4865 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818447    4865 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818451    4865 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818454    4865 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818459    4865 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818462    4865 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818465    4865 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818468    4865 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818471    4865 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818474    4865 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818478    4865 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818481    4865 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818484    4865 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818488    4865 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818491    4865 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818494    4865 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818497    4865 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818500    4865 flags.go:64] FLAG: --enable-server="true"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818503    4865 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818508    4865 flags.go:64] FLAG: --event-burst="100"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818511    4865 flags.go:64] FLAG: --event-qps="50"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818514    4865 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818518    4865 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818525    4865 flags.go:64] FLAG: --eviction-hard=""
Nov 04 01:55:36.825515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818530    4865 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818533    4865 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818536    4865 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818539    4865 flags.go:64] FLAG: --eviction-soft=""
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818542    4865 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818546    4865 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818549    4865 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818553    4865 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818556    4865 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818559    4865 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818562    4865 flags.go:64] FLAG: --feature-gates=""
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818566    4865 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818569    4865 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818572    4865 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818575    4865 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818579    4865 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818582    4865 flags.go:64] FLAG: --help="false"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818586    4865 flags.go:64] FLAG: --hostname-override="ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818589    4865 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818592    4865 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818595    4865 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818599    4865 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818602    4865 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 01:55:36.826160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818606    4865 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818609    4865 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818612    4865 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818615    4865 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818618    4865 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818622    4865 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818625    4865 flags.go:64] FLAG: --kube-reserved=""
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818628    4865 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818633    4865 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818636    4865 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818639    4865 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818642    4865 flags.go:64] FLAG: --lock-file=""
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818645    4865 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818649    4865 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818652    4865 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818658    4865 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818661    4865 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818664    4865 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818667    4865 flags.go:64] FLAG: --logging-format="text"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818671    4865 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818674    4865 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818677    4865 flags.go:64] FLAG: --manifest-url=""
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818680    4865 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818685    4865 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818688    4865 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 01:55:36.826857 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818692    4865 flags.go:64] FLAG: --max-pods="110"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818695    4865 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818698    4865 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818701    4865 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818704    4865 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818707    4865 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818711    4865 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818715    4865 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818722    4865 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818726    4865 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818729    4865 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818732    4865 flags.go:64] FLAG: --pod-cidr=""
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818735    4865 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818738    4865 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818741    4865 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818746    4865 flags.go:64] FLAG: --port="10250"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818749    4865 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818752    4865 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-003e72c3116aa78d7"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818755    4865 flags.go:64] FLAG: --qos-reserved=""
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818758    4865 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818761    4865 flags.go:64] FLAG: --register-node="true"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818764    4865 flags.go:64] FLAG: --register-with-taints=""
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818768    4865 flags.go:64] FLAG: --registry-burst="10"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818771    4865 flags.go:64] FLAG: --registry-qps="5"
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818774    4865 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818778    4865 flags.go:64] FLAG: --reserved-memory=""
Nov 04 01:55:36.827510 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818782    4865 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818785    4865 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818788    4865 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818791    4865 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818794    4865 flags.go:64] FLAG: --runonce="false"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818797    4865 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818800    4865 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818804    4865 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818806    4865 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818810    4865 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818813    4865 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818816    4865 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818819    4865 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818824    4865 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818827    4865 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818830    4865 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818833    4865 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818837    4865 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818840    4865 flags.go:64] FLAG: --system-cgroups=""
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818843    4865 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818848    4865 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818852    4865 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818855    4865 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818859    4865 flags.go:64] FLAG: --tls-min-version=""
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818862    4865 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 01:55:36.828181 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818865    4865 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818868    4865 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818872    4865 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818875    4865 flags.go:64] FLAG: --v="2"
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818879    4865 flags.go:64] FLAG: --version="false"
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818884    4865 flags.go:64] FLAG: --vmodule=""
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818888    4865 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.818892    4865 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819021    4865 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819025    4865 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819029    4865 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819032    4865 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819035    4865 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819038    4865 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819041    4865 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819043    4865 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819046    4865 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819049    4865 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819051    4865 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819054    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819058    4865 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:55:36.828868 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819061    4865 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819064    4865 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819066    4865 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819069    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819072    4865 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819075    4865 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819078    4865 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819081    4865 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819084    4865 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819086    4865 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819089    4865 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819092    4865 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819094    4865 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819097    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819099    4865 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819102    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819105    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819107    4865 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819110    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819113    4865 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:55:36.829441 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819116    4865 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819118    4865 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819121    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819123    4865 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819126    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819129    4865 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819132    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819134    4865 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819137    4865 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819140    4865 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819142    4865 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819146    4865 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819149    4865 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819152    4865 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819155    4865 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819157    4865 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819160    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819162    4865 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819165    4865 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:55:36.830170 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819169    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819173    4865 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819177    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819180    4865 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819183    4865 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819185    4865 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819188    4865 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819191    4865 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819193    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819196    4865 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819198    4865 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819201    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819203    4865 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819207    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819210    4865 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819212    4865 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819215    4865 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819217    4865 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819220    4865 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:55:36.830810 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819223    4865 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819226    4865 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819228    4865 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819231    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819233    4865 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819237    4865 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819240    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819243    4865 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819245    4865 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819248    4865 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819251    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819253    4865 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819256    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819259    4865 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819262    4865 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819264    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819267    4865 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819269    4865 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819272    4865 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819275    4865 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:55:36.831265 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819291    4865 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:55:36.831837 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819294    4865 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:55:36.831837 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819297    4865 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:55:36.831837 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819300    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:55:36.831837 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819302    4865 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:55:36.831837 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819305    4865 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:55:36.831837 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819308    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:55:36.831837 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819311    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:55:36.831837 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819313    4865 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:55:36.831837 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819316    4865 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:55:36.831837 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819318    4865 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:55:36.831837 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819321    4865 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:55:36.831837 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819324    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:55:36.831837 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819328    4865 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:55:36.831837 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819332    4865 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:55:36.831837 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.819358    4865 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819399    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819405    4865 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819413    4865 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819416    4865 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819420    4865 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819424    4865 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819426    4865 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819429    4865 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819432    4865 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819434    4865 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819437    4865 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819440    4865 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819442    4865 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819445    4865 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819448    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819451    4865 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819453    4865 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819456    4865 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:55:36.832360 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819459    4865 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819461    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819464    4865 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819466    4865 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819469    4865 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819471    4865 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819475    4865 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819478    4865 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819481    4865 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819483    4865 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819486    4865 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819488    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819491    4865 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819493    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819496    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819498    4865 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819502    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819505    4865 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819508    4865 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819510    4865 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:55:36.832907 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819513    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819515    4865 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819518    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819521    4865 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819524    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819527    4865 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819530    4865 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819532    4865 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819535    4865 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819538    4865 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819540    4865 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819543    4865 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819545    4865 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819548    4865 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819551    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819553    4865 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819556    4865 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819558    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819562    4865 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819566    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:55:36.833482 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819570    4865 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819573    4865 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819575    4865 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819578    4865 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819581    4865 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819583    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819586    4865 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819588    4865 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819592    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819595    4865 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819598    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819600    4865 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819603    4865 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819606    4865 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819608    4865 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819611    4865 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819614    4865 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819617    4865 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819620    4865 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:55:36.833994 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819622    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819625    4865 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819628    4865 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819630    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819633    4865 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819635    4865 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819638    4865 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819640    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819643    4865 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819645    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819648    4865 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819651    4865 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819653    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819656    4865 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819660    4865 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819663    4865 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819665    4865 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819669    4865 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819672    4865 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819675    4865 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:55:36.834497 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819677    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819681    4865 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819684    4865 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819687    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819689    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819692    4865 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819694    4865 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819697    4865 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.819699    4865 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.823384    4865 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.823399    4865 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.823416    4865 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.823421    4865 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823475    4865 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823479    4865 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823482    4865 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823486    4865 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823489    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823492    4865 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:55:36.835090 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823495    4865 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823498    4865 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823501    4865 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823503    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823506    4865 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823509    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823511    4865 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823515    4865 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823518    4865 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823520    4865 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823523    4865 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823526    4865 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823528    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823531    4865 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823534    4865 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823537    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823540    4865 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823542    4865 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823545    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:55:36.835662 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823548    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823550    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823553    4865 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823556    4865 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823559    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823562    4865 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823565    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823568    4865 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823570    4865 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823573    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823575    4865 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823578    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823580    4865 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823583    4865 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823586    4865 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823588    4865 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823591    4865 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823593    4865 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823596    4865 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823598    4865 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:55:36.836136 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823601    4865 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823604    4865 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823606    4865 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823609    4865 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823611    4865 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823614    4865 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823617    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823619    4865 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823622    4865 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823625    4865 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823627    4865 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823630    4865 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823633    4865 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823635    4865 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823638    4865 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823640    4865 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823643    4865 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823646    4865 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823649    4865 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823651    4865 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:55:36.836635 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823654    4865 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823656    4865 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823659    4865 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823662    4865 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823664    4865 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823667    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823669    4865 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823673    4865 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823678    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823682    4865 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823686    4865 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823688    4865 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823691    4865 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823694    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823696    4865 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823699    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823702    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823704    4865 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823707    4865 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:55:36.837120 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823709    4865 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823712    4865 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823714    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823717    4865 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823719    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823722    4865 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823724    4865 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823727    4865 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823729    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823732    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823735    4865 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823738    4865 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823740    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823743    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823746    4865 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823748    4865 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823751    4865 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823753    4865 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823756    4865 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:55:36.837674 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823758    4865 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:55:36.838146 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823761    4865 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:55:36.838146 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823764    4865 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:55:36.838146 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.823792    4865 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:55:36.838146 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823933    4865 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:55:36.838146 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823937    4865 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:55:36.838146 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823941    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:55:36.838146 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823943    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:55:36.838146 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823946    4865 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:55:36.838146 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823948    4865 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:55:36.838146 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823951    4865 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:55:36.838146 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823954    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:55:36.838146 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823956    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:55:36.838146 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823959    4865 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:55:36.838146 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823961    4865 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:55:36.838146 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823964    4865 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:55:36.838146 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823966    4865 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823969    4865 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823972    4865 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823974    4865 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823976    4865 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823979    4865 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823981    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823984    4865 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823986    4865 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823992    4865 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823995    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.823998    4865 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824001    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824003    4865 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824006    4865 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824008    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824011    4865 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824013    4865 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824016    4865 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824018    4865 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:55:36.838566 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824021    4865 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824023    4865 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824029    4865 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824032    4865 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824034    4865 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824037    4865 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824039    4865 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824042    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824045    4865 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824047    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824050    4865 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824053    4865 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824056    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824058    4865 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824060    4865 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824063    4865 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824065    4865 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824068    4865 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824071    4865 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824073    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:55:36.839059 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824146    4865 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824155    4865 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824711    4865 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824737    4865 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824743    4865 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824749    4865 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824754    4865 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824759    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824764    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824768    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824774    4865 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824779    4865 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824784    4865 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824791    4865 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824822    4865 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824828    4865 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824832    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824837    4865 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824841    4865 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:55:36.839569 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824845    4865 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824850    4865 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824855    4865 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824859    4865 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824863    4865 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824868    4865 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824872    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824876    4865 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824885    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824889    4865 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824894    4865 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824898    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824903    4865 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824907    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824912    4865 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824916    4865 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824921    4865 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824927    4865 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824932    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:55:36.840039 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824936    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:55:36.840549 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824946    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:55:36.840549 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824950    4865 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:55:36.840549 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824955    4865 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:55:36.840549 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824959    4865 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:55:36.840549 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824963    4865 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:55:36.840549 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824967    4865 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:55:36.840549 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824972    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:55:36.840549 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824976    4865 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:55:36.840549 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824981    4865 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:55:36.840549 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824985    4865 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:55:36.840549 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824989    4865 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:55:36.840549 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.824993    4865 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:55:36.840549 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825000    4865 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:55:36.840549 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825014    4865 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:55:36.840549 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825020    4865 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:55:36.841073 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.825085    4865 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:55:36.841073 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825195    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:55:36.841073 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825201    4865 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:55:36.841073 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825205    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:55:36.841073 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825211    4865 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:55:36.841073 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825215    4865 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:55:36.841073 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825220    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:55:36.841073 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825223    4865 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:55:36.841073 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825227    4865 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:55:36.841073 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825231    4865 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:55:36.841073 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825241    4865 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:55:36.841073 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825246    4865 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:55:36.841073 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825251    4865 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:55:36.841073 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825256    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:55:36.841073 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825261    4865 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:55:36.841073 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825266    4865 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825271    4865 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825297    4865 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825302    4865 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825305    4865 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825308    4865 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825311    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825314    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825316    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825320    4865 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825323    4865 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825327    4865 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825333    4865 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825340    4865 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825350    4865 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825356    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825362    4865 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825367    4865 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825371    4865 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:55:36.841749 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825375    4865 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825379    4865 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825384    4865 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825388    4865 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825393    4865 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825398    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825402    4865 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825407    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825415    4865 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825420    4865 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825424    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825428    4865 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825433    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825443    4865 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825613    4865 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825636    4865 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825642    4865 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825647    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825652    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825656    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:55:36.842504 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825661    4865 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825665    4865 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825676    4865 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825681    4865 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825685    4865 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825690    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825696    4865 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825701    4865 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825705    4865 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825709    4865 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825714    4865 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825721    4865 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825727    4865 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825732    4865 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825737    4865 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825742    4865 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825747    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825751    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825755    4865 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:55:36.843034 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825759    4865 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825764    4865 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825768    4865 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825772    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825776    4865 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825780    4865 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825784    4865 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825789    4865 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825793    4865 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825798    4865 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825801    4865 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825806    4865 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825810    4865 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825814    4865 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825818    4865 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825823    4865 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825827    4865 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825831    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825835    4865 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825842    4865 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:55:36.843544 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825847    4865 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825851    4865 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825857    4865 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825862    4865 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825866    4865 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825870    4865 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825874    4865 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825878    4865 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825880    4865 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825883    4865 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825886    4865 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825888    4865 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825891    4865 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:36.825894    4865 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.825965    4865 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.826973    4865 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.827042    4865 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.827234    4865 server.go:1014] "Starting client certificate rotation"
Nov 04 01:55:36.844321 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.827378    4865 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 01:55:36.845139 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.827424    4865 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 01:49:52 +0000 UTC" deadline="2028-06-11 15:00:12.017816661 +0000 UTC"
Nov 04 01:55:36.845139 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.827456    4865 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14053h4m35.190363796s"
Nov 04 01:55:36.845139 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.827796    4865 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:55:36.845139 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.828893    4865 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:55:36.845139 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.830107    4865 log.go:25] "Validated CRI v1 runtime API"
Nov 04 01:55:36.845139 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.831792    4865 log.go:25] "Validated CRI v1 image API"
Nov 04 01:55:36.845139 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.832505    4865 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 01:55:36.845139 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.834306    4865 fs.go:135] Filesystem UUIDs: map[4c658a12-fb8f-4245-a666-190348527387:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b30cc0a9-c0f6-4898-bae5-37d34deda149:/dev/nvme0n1p3]
Nov 04 01:55:36.845555 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.834329    4865 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0aef766ce944d7a0701fee5cc54fb4951ca982cdcdcedf2da8776b68d9f58b07/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0aef766ce944d7a0701fee5cc54fb4951ca982cdcdcedf2da8776b68d9f58b07/userdata/shm major:0 minor:290 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0bb2fc810f7af835c9f906b6fe9baf7bdc2ec668b76db35ffe0481dc28faa034/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0bb2fc810f7af835c9f906b6fe9baf7bdc2ec668b76db35ffe0481dc28faa034/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2b9d2187ceb34aa5ea3b605b593753a274826e1bd5adf4401fcdf9d83949f53b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2b9d2187ceb34aa5ea3b605b593753a274826e1bd5adf4401fcdf9d83949f53b/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/37b91df8e28645c78cae6da6ead79ba0763b96b9e62d36b79cae5566e8a3ab50/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/37b91df8e28645c78cae6da6ead79ba0763b96b9e62d36b79cae5566e8a3ab50/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/39664ccb3f9a6204ddffa1cc3448663e1c16d4aaf62db97d86b97f6ca54b5bb0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/39664ccb3f9a6204ddffa1cc3448663e1c16d4aaf62db97d86b97f6ca54b5bb0/userdata/shm major:0 minor:279 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/429e9d4fd1ba13c323733354917483fb7cac1ec84436db6173e645c1e8c6c66c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/429e9d4fd1ba13c323733354917483fb7cac1ec84436db6173e645c1e8c6c66c/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/435b4c22badc31787ef6c7a4bd26a6b89171efdeca310865c9bd0009b2c51208/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/435b4c22badc31787ef6c7a4bd26a6b89171efdeca310865c9bd0009b2c51208/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4f4681af6a37a15af43b80bc1cde4a59db4649601f60f1bc9e4323af8e8b2b81/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4f4681af6a37a15af43b80bc1cde4a59db4649601f60f1bc9e4323af8e8b2b81/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5ba80f3ee290ca94c7acaf25bd3155ad37d11616addf83d241287ef2a7259bfa/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5ba80f3ee290ca94c7acaf25bd3155ad37d11616addf83d241287ef2a7259bfa/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/629c45f7398cd6c8a2ecb52ed62614f33e8715c09e20a7bceb39ec7706ee998d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/629c45f7398cd6c8a2ecb52ed62614f33e8715c09e20a7bceb39ec7706ee998d/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7b2f600b4ff99de30a4965ee52b741927d150a2474caddea46981c4634af4e12/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7b2f600b4ff99de30a4965ee52b741927d150a2474caddea46981c4634af4e12/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7f966c4979ac588dd8a85320db55bac8c3612d048637276dc06f572e05654ea7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7f966c4979ac588dd8a85320db55bac8c3612d048637276dc06f572e05654ea7/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8affe663e5e234923e78515c2f437f227257c62324a262f64ddbab5ca6a92bff/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8affe663e5e234923e78515c2f437f227257c62324a262f64ddbab5ca6a92bff/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9cb410ae7eb6035bbc16b4def3d7faf5c3582513da295e519a4751b2a7d761f6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9cb410ae7eb6035bbc16b4def3d7faf5c3582513da295e519a4751b2a7d761f6/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a05fdbadc57b9071e6a89a56a74230395c051348837b1d9588334d308a1d1132/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a05fdbadc57b9071e6a89a56a74230395c051348837b1d9588334d308a1d1132/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a4cf7f99d75dbefad68a1a3ecfe1f4bc0aa617025acd7fe6bc3aa4552455a86a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a4cf7f99d75dbefad68a1a3ecfe1f4bc0aa617025acd7fe6bc3aa4552455a86a/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b101aa382934986eaf277925805516432fd4e9bc320282679775888f7a46b1a1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b101aa382934986eaf277925805516432fd4e9bc320282679775888f7a46b1a1/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c7debcb3ff65b6f537be805308349656b4ce200749278be121b7d361958794b5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c7debcb3ff65b6f537be805308349656b4ce200749278be121b7d361958794b5/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c90dfcd721e4bf20cab1e57beb6127a78cb16c800014c20d17192be40af26ba5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c90dfcd721e4bf20cab1e57beb6127a78cb16c800014c20d17192be40af26ba5/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d6b0d38df1a9dc473b2721487dca1b06108f5eb26aee7a939748f3a0be7b4e86/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d6b0d38df1a9dc473b2721487dca1b06108f5eb26aee7a939748f3a0be7b4e86/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e203d9f7f4dace84a1605cb1848bc64e2806a328e3bf76639b08b554fc584054/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e203d9f7f4dace84a1605cb1848bc64e2806a328e3bf76639b08b554fc584054/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f51632b5c6188bd247ed22b62c42f49a22426c075ccc595088ba3744454f5aa8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f51632b5c6188bd247ed22b62c42f49a22426c075ccc595088ba3744454f5aa8/userdata/shm major:0 minor:211 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aa5b60a-ce95-4623-8ae2-32896ca1232d/volumes/kubernetes.io~projected/kube-api-access-4sfr5:{mountpoint:/var/lib/kubelet/pods/0aa5b60a-ce95-4623-8ae2-32896ca1232d/volumes/kubernetes.io~projected/kube-api-access-4sfr5 major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1536e272-39b6-4702-a35d-0130eef3e290/volumes/kubernetes.io~projected/kube-api-access-8trls:{mountpoint:/var/lib/kubelet/pods/1536e272-39b6-4702-a35d-0130eef3e290/volumes/kubernetes.io~projected/kube-api-access-8trls major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1c13ee40-c34c-4575-8865-aceeb71759d0/volumes/kubernetes.io~projected/kube-api-access-2c8q8:{mountpoint:/var/lib/kubelet/pods/1c13ee40-c34c-4575-8865-aceeb71759d0/volumes/kubernetes.io~projected/kube-api-access-2c8q8 major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1d895e08-f347-4b7c-b63e-26b06a3f6c10/volumes/kubernetes.io~projected/kube-api-access-28wg4:{mountpoint:/var/lib/kubelet/pods/1d895e08-f347-4b7c-b63e-26b06a3f6c10/volumes/kubernetes.io~projected/kube-api-access-28wg4 major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/213808a8-b7ec-4fc6-845b-d2648393d5bc/volumes/kubernetes.io~projected/kube-api-access-56z2g:{mountpoint:/var/lib/kubelet/pods/213808a8-b7ec-4fc6-845b-d2648393d5bc/volumes/kubernetes.io~projected/kube-api-access-56z2g major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2522cd43-48a3-40d4-9f8f-b7a0f778abbd/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/2522cd43-48a3-40d4-9f8f-b7a0f778abbd/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2522cd43-48a3-40d4-9f8f-b7a0f778abbd/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/2522cd43-48a3-40d4-9f8f-b7a0f778abbd/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2522cd43-48a3-40d4-9f8f-b7a0f778abbd/volumes/kubernetes.io~projected/kube-api-access-gdkv4:{mountpoint:/var/lib/kubelet/pods/2522cd43-48a3-40d4-9f8f-b7a0f778abbd/volumes/kubernetes.io~projected/kube-api-access-gdkv4 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30efbdba-8299-418b-b8f1-233c789443f5/volumes/kubernetes.io~projected/kube-api-access-lxwj7:{mountpoint:/var/lib/kubelet/pods/30efbdba-8299-418b-b8f1-233c789443f5/volumes/kubernetes.io~projected/kube-api-access-lxwj7 major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40363d65-7ece-4f05-8668-439a60582a18/volumes/kubernetes.io~projected/kube-api-access-b655g:{mountpoint:/var/lib/kubelet/pods/40363d65-7ece-4f05-8668-439a60582a18/volumes/kubernetes.io~projected/kube-api-access-b655g major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/417744b4-5ed7-4b20-9960-7147d2d567ee/volumes/kubernetes.io~projected/kube-api-access-nlzhr:{mountpoint:/var/lib/kubelet/pods/417744b4-5ed7-4b20-9960-7147d2d567ee/volumes/kubernetes.io~projected/kube-api-access-nlzhr major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/417744b4-5ed7-4b20-9960-7147d2d567ee/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/417744b4-5ed7-4b20-9960-7147d2d567ee/volumes/kubernetes.io~secret/serving-cert major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4d14f606-6852-4eff-b71f-5a8adcde5f84/volumes/kubernetes.io~projected/kube-api-access-v2c2n:{mountpoint:/var/lib/kubelet/pods/4d14f606-6852-4eff-b71f-5a8adcde5f84/volumes/kubernetes.io~projected/kube-api-access-v2c2n major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/698e630d-a5b0-43ce-9a12-aac9c60ab53c/volumes/kubernetes.io~projected/kube-api-access-fppqt:{mountpoint:/var/lib/kubelet/pods/698e630d-a5b0-43ce-9a12-aac9c60ab53c/volumes/kubernetes.io~projected/kube-api-access-fppqt major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/698e630d-a5b0-43ce-9a12-aac9c60ab53c/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/698e630d-a5b0-43ce-9a12-aac9c60ab53c/volumes/kubernetes.io~secret/serving-cert major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/74e217e6-636d-4bab-b37f-c44dc0d8fe81/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/74e217e6-636d-4bab-b37f-c44dc0d8fe81/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/74e217e6-636d-4bab-b37f-c44dc0d8fe81/volumes/kubernetes.io~projected/kube-api-access-k9hwp:{mountpoint:/var/lib/kubelet/pods/74e217e6-636d-4bab-b37f-c44dc0d8fe81/volumes/kubernetes.io~projected/kube-api-access-k9hwp major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/74e217e6-636d-4bab-b37f-c44dc0d8fe81/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/74e217e6-636d-4bab-b37f-c44dc0d8fe81/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/74e217e6-636d-4bab-b37f-c44dc0d8fe81/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/74e217e6-636d-4bab-b37f-c44dc0d8fe81/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/789e64a1-501d-46d9-a58d-14f786410301/volumes/kubernetes.io~projected/kube-api-access-78j5f:{mountpoint:/var/lib/kubelet/pods/789e64a1-501d-46d9-a58d-14f786410301/volumes/kubernetes.io~projected/kube-api-access-78j5f major:0 minor:312 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/870962af-eac0-484d-9267-db8ab42ef13d/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/870962af-eac0-484d-9267-db8ab42ef13d/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/870962af-eac0-484d-9267-db8ab42ef13d/volumes/kubernetes.io~projected/kube-api-access-9g6rr:{mountpoint:/var/lib/kubelet/pods/870962af-eac0-484d-9267-db8ab42ef13d/volumes/kubernetes.io~projected/kube-api-access-9g6rr major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/870962af-eac0-484d-9267-db8ab42ef13d/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/870962af-eac0-484d-9267-db8ab42ef13d/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/870962af-eac0-484d-9267-db8ab42ef13d/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/870962af-eac0-484d-9267-db8ab42ef13d/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/99ef12a9-a676-4258-a3a8-e7fcd611d158/volumes/kubernetes.io~projected/kube-api-access-k8qmb:{mountpoint:/var/lib/kubelet/pods/99ef12a9-a676-4258-a3a8-e7fcd611d158/volumes/kubernetes.io~projected/kube-api-access-k8qmb major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13/volumes/kubernetes.io~projected/kube-api-access-nbn5k:{mountpoint:/var/lib/kubelet/pods/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13/volumes/kubernetes.io~projected/kube-api-access-nbn5k major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a8ecf9a5-a6e5-460b-8cc9-44c996d57930/volumes/kubernetes.io~projected/kube-api-access-f68tp:{mountpoint:/var/lib/kubelet/pods/a8ecf9a5-a6e5-460b-8cc9-44c996d57930/volumes/kubernetes.io~projected/kube-api-access-f68tp major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a8ecf9a5-a6e5-460b-8cc9-44c996d57930/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/a8ecf9a5-a6e5-460b-8cc9-44c996d57930/volumes/kubernetes.io~secret/serving-cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a994b38f-0096-4fff-b49c-e6d99dc5a6be/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/a994b38f-0096-4fff-b49c-e6d99dc5a6be/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:278 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b1926369-9621-4cac-b797-43a306bdb709/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/b1926369-9621-4cac-b797-43a306bdb709/volumes/kubernetes.io~secret/agent-certs major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b2a74b29-e9ea-4ab9-b155-795d08684fe6/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/b2a74b29-e9ea-4ab9-b155-795d08684fe6/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b2a74b29-e9ea-4ab9-b155-795d08684fe6/volumes/kubernetes.io~projected/kube-api-access-c8lmb:{mountpoint:/var/lib/kubelet/pods/b2a74b29-e9ea-4ab9-b155-795d08684fe6/volumes/kubernetes.io~projected/kube-api-access-c8lmb major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b2a74b29-e9ea-4ab9-b155-795d08684fe6/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/b2a74b29-e9ea-4ab9-b155-795d08684fe6/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b461004a-433c-4411-84df-b9aea9f682ab/volumes/kubernetes.io~projected/kube-api-access-44d8q:{mountpoint:/var/lib/kubelet/pods/b461004a-433c-4411-84df-b9aea9f682ab/volumes/kubernetes.io~projected/kube-api-access-44d8q major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c07ce523-d075-4203-be44-3f7d5af7e0d7/volumes/kubernetes.io~projected/kube-api-access-xv54m:{mountpoint:/var/lib/kubelet/pods/c07ce523-d075-4203-be44-3f7d5af7e0d7/volumes/kubernetes.io~projected/kube-api-access-xv54m major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c73834fc-41c8-4eea-a739-52d16535890d/volumes/kubernetes.io~projected/kube-api-access-9tjn7:{mountpoint:/var/lib/kubelet/pods/c73834fc-41c8-4eea-a739-52d16535890d/volumes/kubernetes.io~projected/kube-api-access-9tjn7 major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971/volumes/kubernetes.io~projected/kube-api-access-mcz47:{mountpoint:/var/lib/kubelet/pods/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971/volumes/kubernetes.io~projected/kube-api-access-mcz47 major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971/volumes/kubernetes.io~secret/default-certificate major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971/volumes/kubernetes.io~secret/stats-auth major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d09416ce-b095-4a28-9cb9-c0ad486ec03c/volumes/kubernetes.io~projected/kube-api-access-kslbj:{mountpoint:/var/lib/kubelet/pods/d09416ce-b095-4a28-9cb9-c0ad486ec03c/volumes/kubernetes.io~projected/kube-api-access-kslbj major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d09416ce-b095-4a28-9cb9-c0ad486ec03c/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/d09416ce-b095-4a28-9cb9-c0ad486ec03c/volumes/kubernetes.io~secret/serving-cert major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eddb65a3-0fa8-48b5-8ec9-4c939e189be4/volumes/kubernetes.io~projected/kube-api-access-rmpkm:{mountpoint:/var/lib/kubelet/pods/eddb65a3-0fa8-48b5-8ec9-4c939e189be4/volumes/kubernetes.io~projected/kube-api-access-rmpkm major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ef75aae9-bb5f-4e09-b374-fa398aadfae9/volumes/kubernetes.io~projected/kube-api-access-mrt5f:{mountpoint:/var/lib/kubelet/pods/ef75aae9-bb5f-4e09-b374-fa398aadfae9/volumes/kubernetes.io~projected/kube-api-access-mrt5f major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f3db2db6-6f0f-4af7-9bb7-493df9107bb9/volumes/kubernetes.io~projected/kube-api-access-2xrfz:{mountpoint:/var/lib/kubelet/pods/f3db2db6-6f0f-4af7-9bb7-493df9107bb9/volumes/kubernetes.io~projected/kube-api-access-2xrfz major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f3db2db6-6f0f-4af7-9bb7-493df9107bb9/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/f3db2db6-6f0f-4af7-9bb7-493df9107bb9/volumes/kubernetes.io~secret/cert major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ff8251b4-1f23-4a86-9b1d-602c87cf133b/volumes/kubernetes.io~projected/kube-api-access-885ds:{mountpoint:/var/lib/kubelet/pods/ff8251b4-1f23-4a86-9b1d-602c87cf133b/volumes/kubernetes.io~projected/kube-api-access-885ds major:0 minor:57 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/0da24a5d8dfdbfb66cbbd84eefd02761ab811a98cb5d5339630a289e3ee80add/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/2bc453c0078b1a0904cf873d1f4566fb4b478d1dfbe152de4f9f9a40aabc4501/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/e38cbd21742c1c87eb365f4fd3b52b6905709550bbeddf9ec4348fbb0f44d75a/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/5304553040e5ccaf832635a0e3216f1b5c20b1ee5fbf7cef91f56c2634272d6d/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/2419c57e266e0de63cd1af4cdef972ab7994a21234075c2a3c75e3912f153658/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/6cc71e2f823c3426550e2a3eb551c4b9bd1b198bc364f7bb7b40d12afd5c6f79/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-121:{mountpoint:/var/lib/containers/storage/overlay/edd4fdebd78acf80f05f8f7c549b1e0e376641874d346d8a1ac9469b44e1f3ca/merged major:0 minor:121 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/420fcfa9a93dd5f96fae635a076718325a59acdc0cfed1e9b4c278e05abe0665/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/5b5c45721fb5fab8ba946b50a6a3dc70a5fc39ea1bc023728a36c1564ec74e14/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/eeb3f011597668ce47871ff71814bf39a1fb92ff4da5e910ea2ec5c5ecf847e5/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-136:{mountpoint:/var/lib/containers/storage/overlay/486de388ee5fdbbf72ffa2bc5f988078e540a170e7ad4c697a4656d865bb930f/merged major:0 minor:136 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/a6f804aadeb06e1c64cffaea9c2c62144952ab0ef89307513b4ea50d0c92dd60/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/cea64ff3d19fe822eea285f59e69c4cf7fbb187c93ffb28d45de8753aa0ee635/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/78c4059c1ba67c0d154b20a581aba0dd1d04ceeadeb1b3ad72bd797af977c9a7/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/4380c84d16412c496c8c2eec96b8d0abd3efd3ea73b8bdad322d1144915f2af0/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/6874b2fec77dabac2ebdfc7ae6d004968bcd98355fba488f16fd4a34714162a6/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/d8057fb46fcdb5a8054825b24ada281302343607ac337494ad57aa3652505763/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/d2dafa945e8a20b9a975bac3d5b9f0976f0aad5a7879b3fe0840cd19dc7f8b98/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/142e059ac61be07e72007cd07b9de2f3fafb14376f24c20b455cd1899831625c/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/e005ad2a10377a58e5b286816d366b57f05ca9e5a7ec6176559357a867a9f45c/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/b945672173a75b9238a753f2dca1de12314b5e1670e664e411277ec02a1cc8f3/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/8cf823b6c27f2501e61f9a93a790809f264c5122f7f8db225e231a68d1cb5e64/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/7578cf53ec5c4ea11622cac9339dad1991de37d2a5858ae9d57dc2e71910c61e/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/37f32ea903cac883e4b81b419a12bb2bae02ad110495e0ab626976de01449307/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/3586bbc70b7cada84d37a58a7ba503683d4286c6578f38a06aa6999521cf55a5/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/6a917f5b0cf927b54abb2a69e83b4cf41e9d61734c8f36a7831e4a0cfb84b384/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-257:{mountpoint:/var/lib/containers/storage/overlay/dfcb94db7d33d679a470c8006dc8ba34e54650e784e6c3c4042fe0ed0d906bfb/merged major:0 minor:257 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/f5010b915c3914b06c6c99eaa53989a9a6b4b2a82025abde82c49af4ed799780/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-265:{mountpoint:/var/lib/containers/storage/overlay/8a38bff38f74d26f6ace58301449bb20f730765fa434c4d97a765db614d70a12/merged major:0 minor:265 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/b6edda45dc7f13de07784aeea67d3864c991b7ffc2283e5cdd7d7d3962b2d13a/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/656ccc3778632a7d059d474926aedb854d0913bd0f3526753217f903ef66cbc1/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/f7d9c83084f9ae399e2da49f4ba42caedae23d0370371ff3814dacba575e0ae7/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-274:{mountpoint:/var/lib/containers/storage/overlay/f54a8be1c8c279dcbe9c4a7f054cbbf86a304a976e48272c21f5bb684cc5fecc/merged major:0 minor:274 fsType:overlay blockSize:0} overlay_0-276:{mountpoint:/var/lib/containers/storage/overlay/9d62385cd44915dd68a211d41ce8b6394f64fdb70922a784fc0a969ae1a9e917/merged major:0 minor:276 fsType:overlay blockSize:0} overlay_0-292:{mountpoint:/var/lib/containers/storage/overlay/72046e06bf6f2f0fc0aed6611bab525fedd2d4f7c925613e8cd0249a95328589/merged major:0 minor:292 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/6b946a764ccbe72f1466f1e55ef73a53775c33619790b165a29e468744ef45b7/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/28e27a6fb54e22bd5283def5eb4c9c9035f9bdae9a142be1fca6bdfb9575cfdd/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/49540d6c60a01911c8613be9b4485dc14ef0af25525f7097b953b728cecc14fb/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/44d7fe97b033c4b4ddc68962b60aa9cee8f9c8dd7eb77aa4b4e3861216ca7bc6/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/43e64f31a5994fca65919b99633b3c8919ad6ab0e7b6fd3a9fa1a3da61e663b9/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-317:{mountpoint:/var/lib/containers/storage/overlay/f8c810fa89dff9bc873a7f9309a5f088609a93f7b3aa9457e67fd9b08ad55231/merged major:0 minor:317 fsType:overlay blockSize:0} overlay_0-323:{mountpoint:/var/lib/containers/storage/overlay/ddff1f80f19bc644e1b7842846bb50c752198d23e6fa16efdeffb9c6d6522109/merged major:0 minor:323 fsType:overlay blockSize:0} overlay_0-329:{mountpoint:/var/lib/containers/storage/overlay/cbe10a7f69aab541dcc6dc20e0b2bd6dc42ae6f5e4d7600170691f9e4267ee70/merged major:0 minor:329 fsType:overlay blockSize:0} overlay_0-331:{mountpoint:/var/lib/containers/storage/overlay/9d9259fd0e8c6d6a9a72c95a83a8740ebcb483300a9e6ad4ca7f5bebacb0e0ed/merged major:0 minor:331 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/9f39b6207ff23934d57d80459a001c907d34a8d475757453663dc6749dac30df/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/0ab834f0c0e7ded267b45cd1681274b7758bcf20a579b52761b8519056027568/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/1aaa26d9025d19cec69b64e9e6d5a8800d4fdb34671756ea0357358594832feb/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/2db3e08d7c559eb03c165b84d9fe1974990c3ac06e3551f65d8d64b08a339de6/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/31bf2e71f9125930c34e31634369fc2b08d4e587792ac1dc39e3d7c4e952ae76/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/fbba2b5c8144a5162339cdb7f50e82720ff9971dea3fae8778a0831e7e876301/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/a9969fe0b16b8dc5b5675a849b75ca381cb5bbc6a8197b39e87a4f1378eb58e1/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/ba07501f053cfd6d1df0157b6c079065434e91e3582a387eaf949b768a25f672/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/a2cf2e739932d7b5ea6ddbb1d6a63bfeddc89c8b2302e4852794d8dc56c8674d/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/ff0ca1c76a696b4c8f387f61c6f28e3f4a53de2ac0c96321a41862d011dd701f/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.842351    4865 manager.go:217] Machine: {Timestamp:2026-11-04 01:55:36.841460256 +0000 UTC m=+0.068246920 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103790 MemoryCapacity:33164324864 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ff0932fd495d05d5e309ac8fcf97e SystemUUID:ec2ff093-2fd4-95d0-5d5e-309ac8fcf97e BootID:2f49a374-6381-48f4-b347-391da6f491c3 Filesystems:[{Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-265 DeviceMajor:0 DeviceMinor:265 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/213808a8-b7ec-4fc6-845b-d2648393d5bc/volumes/kubernetes.io~projected/kube-api-access-56z2g DeviceMajor:0 DeviceMinor:289 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4f4681af6a37a15af43b80bc1cde4a59db4649601f60f1bc9e4323af8e8b2b81/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-257 DeviceMajor:0 DeviceMinor:257 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-323 DeviceMajor:0 DeviceMinor:323 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/435b4c22badc31787ef6c7a4bd26a6b89171efdeca310865c9bd0009b2c51208/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-329 DeviceMajor:0 DeviceMinor:329 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2522cd43-48a3-40d4-9f8f-b7a0f778abbd/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/b2a74b29-e9ea-4ab9-b155-795d08684fe6/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:60 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-121 DeviceMajor:0 DeviceMinor:121 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4d14f606-6852-4eff-b71f-5a8adcde5f84/volumes/kubernetes.io~projected/kube-api-access-v2c2n DeviceMajor:0 DeviceMinor:223 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-274 DeviceMajor:0 DeviceMinor:274 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/629c45f7398cd6c8a2ecb52ed62614f33e8715c09e20a7bceb39ec7706ee998d/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/870962af-eac0-484d-9267-db8ab42ef13d/volumes/kubernetes.io~projected/kube-api-access-9g6rr DeviceMajor:0 DeviceMinor:214 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/d09416ce-b095-4a28-9cb9-c0ad486ec03c/volumes/kubernetes.io~projected/kube-api-access-kslbj DeviceMajor:0 DeviceMinor:226 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5ba80f3ee290ca94c7acaf25bd3155ad37d11616addf83d241287ef2a7259bfa/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/789e64a1-501d-46d9-a58d-14f786410301/volumes/kubernetes.io~projected/kube-api-access-78j5f DeviceMajor:0 DeviceMinor:312 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/30efbdba-8299-418b-b8f1-233c789443f5/volumes/kubernetes.io~projected/kube-api-access-lxwj7 DeviceMajor:0 DeviceMinor:58 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/0aa5b60a-ce95-4623-8ae2-32896ca1232d/volumes/kubernetes.io~projected/kube-api-access-4sfr5 DeviceMajor:0 DeviceMinor:63 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:205 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-331 DeviceMajor:0 DeviceMinor:331 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8affe663e5e234923e78515c2f437f227257c62324a262f64ddbab5ca6a92bff/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f3db2db6-6f0f-4af7-9bb7-493df9107bb9/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:209 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/ef75aae9-bb5f-4e09-b374-fa398aadfae9/volumes/kubernetes.io~projected/kube-api-access-mrt5f DeviceMajor:0 DeviceMinor:217 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7f966c4979ac588dd8a85320db55bac8c3612d048637276dc06f572e05654ea7/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/40363d65-7ece-4f05-8668-439a60582a18/volumes/kubernetes.io~projected/kube-api-access-b655g DeviceMajor:0 DeviceMinor:228 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/429e9d4fd1ba13c323733354917483fb7cac1ec84436db6173e645c1e8c6c66c/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0aef766ce944d7a0701fee5cc54fb4951ca982cdcdcedf2da8776b68d9f58b07/userdata/shm DeviceMajor:0 DeviceMinor:290 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/c07ce523-d075-4203-be44-3f7d5af7e0d7/volumes/kubernetes.io~projected/kube-api-access-xv54m DeviceMajor:0 DeviceMinor:45 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/37b91df8e28645c78cae6da6ead79ba0763b96b9e62d36b79cae5566e8a3ab50/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/74e217e6-636d-4bab-b37f-c44dc0d8fe81/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:210 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971/volumes/kubernetes.io~projected/kube-api-access-mcz47 DeviceMajor:0 DeviceMinor:215 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/870962af-eac0-484d-9267-db8ab42ef13d/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:218 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d6b0d38df1a9dc473b2721487dca1b06108f5eb26aee7a939748f3a0be7b4e86/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/f3db2db6-6f0f-4af7-9bb7-493df9107bb9/volumes/kubernetes.io~projected/kube-api-access-2xrfz DeviceMajor:0 DeviceMinor:219 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c7debcb3ff65b6f537be805308349656b4ce200749278be121b7d361958794b5/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/39664ccb3f9a6204ddffa1cc3448663e1c16d4aaf62db97d86b97f6ca54b5bb0/userdata/shm DeviceMajor:0 DeviceMinor:279 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-317 DeviceMajor:0 DeviceMinor:317 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a8ecf9a5-a6e5-460b-8cc9-44c996d57930/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:201 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b1926369-9621-4cac-b797-43a306bdb709/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:61 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/b2a74b29-e9ea-4ab9-b155-795d08684fe6/volumes/kubernetes.io~projected/kube-api-access-c8lmb DeviceMajor:0 DeviceMinor:64 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/870962af-eac0-484d-9267-db8ab42ef13d/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:203 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f51632b5c6188bd247ed22b62c42f49a22426c075ccc595088ba3744454f5aa8/userdata/shm DeviceMajor:0 DeviceMinor:211 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/1d895e08-f347-4b7c-b63e-26b06a3f6c10/volumes/kubernetes.io~projected/kube-api-access-28wg4 DeviceMajor:0 DeviceMinor:65 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/1536e272-39b6-4702-a35d-0130eef3e290/volumes/kubernetes.io~projected/kube-api-access-8trls DeviceMajor:0 DeviceMinor:66 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1c13ee40-c34c-4575-8865-aceeb71759d0/volumes/kubernetes.io~projected/kube-api-access-2c8q8 DeviceMajor:0 DeviceMinor:220 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a4cf7f99d75dbefad68a1a3ecfe1f4bc0aa617025acd7fe6bc3aa4552455a86a/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/2522cd43-48a3-40d4-9f8f-b7a0f778abbd/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:55 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d09416ce-b095-4a28-9cb9-c0ad486ec03c/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:197 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/b461004a-433c-4411-84df-b9aea9f682ab/volumes/kubernetes.io~projected/kube-api-access-44d8q DeviceMajor:0 DeviceMinor:225 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2522cd43-48a3-40d4-9f8f-b7a0f778abbd/volumes/kubernetes.io~projected/kube-api-access-gdkv4 DeviceMajor:0 DeviceMinor:59 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/74e217e6-636d-4bab-b37f-c44dc0d8fe81/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:202 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-292 DeviceMajor:0 DeviceMinor:292 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-136 DeviceMajor:0 DeviceMinor:136 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/698e630d-a5b0-43ce-9a12-aac9c60ab53c/volumes/kubernetes.io~projected/kube-api-access-fppqt DeviceMajor:0 DeviceMinor:221 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/698e630d-a5b0-43ce-9a12-aac9c60ab53c/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:204 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/417744b4-5ed7-4b20-9960-7147d2d567ee/volumes/kubernetes.io~projected/kube-api-access-nlzhr DeviceMajor:0 DeviceMinor:213 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7b2f600b4ff99de30a4965ee52b741927d150a2474caddea46981c4634af4e12/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e203d9f7f4dace84a1605cb1848bc64e2806a328e3bf76639b08b554fc584054/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/870962af-eac0-484d-9267-db8ab42ef13d/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:207 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/74e217e6-636d-4bab-b37f-c44dc0d8fe81/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:222 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c90dfcd721e4bf20cab1e57beb6127a78cb16c800014c20d17192be40af26ba5/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/74e217e6-636d-4bab-b37f-c44dc0d8fe81/volumes/kubernetes.io~projected/kube-api-access-k9hwp DeviceMajor:0 DeviceMinor:227 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b2a74b29-e9ea-4ab9-b155-795d08684fe6/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/eddb65a3-0fa8-48b5-8ec9-4c939e189be4/volumes/kubernetes.io~projected/kube-api-access-rmpkm DeviceMajor:0 DeviceMinor:224 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2b9d2187ceb34aa5ea3b605b593753a274826e1bd5adf4401fcdf9d83949f53b/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a05fdbadc57b9071e6a89a56a74230395c051348837b1d9588334d308a1d1132/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c73834fc-41c8-4eea-a739-52d16535890d/volumes/kubernetes.io~projected/kube-api-access-9tjn7 DeviceMajor:0 DeviceMinor:67 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b101aa382934986eaf277925805516432fd4e9bc320282679775888f7a46b1a1/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9cb410ae7eb6035bbc16b4def3d7faf5c3582513da295e519a4751b2a7d761f6/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/99ef12a9-a676-4258-a3a8-e7fcd611d158/volumes/kubernetes.io~projected/kube-api-access-k8qmb DeviceMajor:0 DeviceMinor:229 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582160384 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/ff8251b4-1f23-4a86-9b1d-602c87cf133b/volumes/kubernetes.io~projected/kube-api-access-885ds DeviceMajor:0 DeviceMinor:57 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/417744b4-5ed7-4b20-9960-7147d2d567ee/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:206 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/a8ecf9a5-a6e5-460b-8cc9-44c996d57930/volumes/kubernetes.io~projected/kube-api-access-f68tp DeviceMajor:0 DeviceMinor:216 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a994b38f-0096-4fff-b49c-e6d99dc5a6be/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:278 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:208 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-276 DeviceMajor:0 DeviceMinor:276 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13/volumes/kubernetes.io~projected/kube-api-access-nbn5k DeviceMajor:0 DeviceMinor:62 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0bb2fc810f7af835c9f906b6fe9baf7bdc2ec668b76db35ffe0481dc28faa034/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0aef766ce944d7a MacAddress:1a:cb:46:22:b3:44 Speed:10000 Mtu:8901} {Name:0bb2fc810f7af83 MacAddress:ce:d7:b9:ca:6d:1a Speed:10000 Mtu:8901} {Name:2b9d2187ceb34aa MacAddress:06:30:2a:a6:a7:03 Speed:10000 Mtu:8901} {Name:39664ccb3f9a620 MacAddress:b6:12:13:9b:55:16 Speed:10000 Mtu:8901} {Name:429e9d4fd1ba13c MacAddress:12:54:7c:5e:71:ef Speed:10000 Mtu:8901} {Name:435b4c22badc317 MacAddress:9e:73:94:4d:9a:c0 Speed:10000 Mtu:8901} {Name:5ba80f3ee290ca9 MacAddress:0a:19:08:fd:39:d8 Speed:10000 Mtu:8901} {Name:7b2f600b4ff99de MacAddress:e6:bb:e0:9e:c6:6d Speed:10000 Mtu:8901} {Name:a05fdbadc57b907 MacAddress:0e:f6:ca:90:a4:8b Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:78:3c:10:b8:ab Speed:0 Mtu:9001} {Name:br-int MacAddress:06:ea:23:a6:57:b1 Speed:0 Mtu:8901} {Name:c7debcb3ff65b6f MacAddress:86:26:c1:b2:50:b0 Speed:10000 Mtu:8901} {Name:d6b0d38df1a9dc4 MacAddress:8a:ac:88:23:05:0a Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:78:3c:10:b8:ab Speed:0 Mtu:9001} {Name:f51632b5c6188bd MacAddress:6a:0a:5d:c5:74:a6 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:ba:27:c0:64:7e:d9 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:b2:3b:28:79:b5:d3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164324864 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.842693    4865 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.842740    4865 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.843000    4865 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.843018    4865 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-204.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.843175    4865 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.843183    4865 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.843193    4865 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.843214    4865 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.843232    4865 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.843399    4865 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.843522    4865 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.843556    4865 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.843574    4865 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.843591    4865 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.843605    4865 kubelet.go:405] "Adding apiserver pod source"
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.843616    4865 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.844469    4865 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.844479    4865 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:55:36.845913 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.844924    4865 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 01:55:36.847194 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847178    4865 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 01:55:36.847194 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847191    4865 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:55:36.847320 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847305    4865 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 04 01:55:36.847522 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847507    4865 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 01:55:36.847584 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847538    4865 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 01:55:36.847651 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847640    4865 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 01:55:36.847702 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847656    4865 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 01:55:36.847702 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847662    4865 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 01:55:36.847702 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847669    4865 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 01:55:36.847702 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847688    4865 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 01:55:36.847702 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847699    4865 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 01:55:36.847915 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847706    4865 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 01:55:36.847915 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847712    4865 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 01:55:36.847915 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847719    4865 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 01:55:36.847915 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847725    4865 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 01:55:36.847915 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847744    4865 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 01:55:36.847915 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847755    4865 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 01:55:36.847915 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847782    4865 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 01:55:36.847915 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.847790    4865 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 01:55:36.851271 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.851256    4865 server.go:1263] "Started kubelet"
Nov 04 01:55:36.851506 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.851363    4865 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 01:55:36.851853 ip-10-0-134-204 systemd[1]: Started Kubernetes Kubelet.
Nov 04 01:55:36.853060 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.851866    4865 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 01:55:36.853162 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.853098    4865 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 01:55:36.854016 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.853834    4865 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 01:55:36.856371 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.854894    4865 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 01:55:36.856371 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.856251    4865 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 01:55:36.856371 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.856339    4865 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 01:55:36.856569 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.856485    4865 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 01:49:54 +0000 UTC" deadline="2028-04-02 12:48:53.489595861 +0000 UTC"
Nov 04 01:55:36.856569 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.856520    4865 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12370h53m16.633078973s"
Nov 04 01:55:36.857072 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:36.856752    4865 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:55:36.857072 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.856816    4865 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 01:55:36.857072 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.856832    4865 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 01:55:36.857072 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.856977    4865 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 01:55:36.860335 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.858923    4865 factory.go:55] Registering systemd factory
Nov 04 01:55:36.860335 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.858946    4865 factory.go:223] Registration of the systemd container factory successfully
Nov 04 01:55:36.860335 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.859327    4865 factory.go:153] Registering CRI-O factory
Nov 04 01:55:36.860335 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.859371    4865 factory.go:223] Registration of the crio container factory successfully
Nov 04 01:55:36.860335 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.859459    4865 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 01:55:36.860335 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.859478    4865 factory.go:103] Registering Raw factory
Nov 04 01:55:36.860335 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.859493    4865 manager.go:1199] Started watching for new ooms in manager
Nov 04 01:55:36.860335 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.859831    4865 manager.go:319] Starting recovery of all containers
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861498    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40363d65-7ece-4f05-8668-439a60582a18" volumeName="kubernetes.io/configmap/40363d65-7ece-4f05-8668-439a60582a18-telemetry-config" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861564    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74e217e6-636d-4bab-b37f-c44dc0d8fe81" volumeName="kubernetes.io/empty-dir/74e217e6-636d-4bab-b37f-c44dc0d8fe81-ca-trust-extracted" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861581    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b2a74b29-e9ea-4ab9-b155-795d08684fe6" volumeName="kubernetes.io/secret/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861595    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1536e272-39b6-4702-a35d-0130eef3e290" volumeName="kubernetes.io/projected/1536e272-39b6-4702-a35d-0130eef3e290-kube-api-access-8trls" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861608    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40363d65-7ece-4f05-8668-439a60582a18" volumeName="kubernetes.io/projected/40363d65-7ece-4f05-8668-439a60582a18-kube-api-access-b655g" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861622    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="417744b4-5ed7-4b20-9960-7147d2d567ee" volumeName="kubernetes.io/configmap/417744b4-5ed7-4b20-9960-7147d2d567ee-config" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861634    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="789e64a1-501d-46d9-a58d-14f786410301" volumeName="kubernetes.io/empty-dir/789e64a1-501d-46d9-a58d-14f786410301-data-volume" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861646    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="870962af-eac0-484d-9267-db8ab42ef13d" volumeName="kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-registry-certificates" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861661    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b2a74b29-e9ea-4ab9-b155-795d08684fe6" volumeName="kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovnkube-config" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861680    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b2a74b29-e9ea-4ab9-b155-795d08684fe6" volumeName="kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovnkube-script-lib" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861693    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d09416ce-b095-4a28-9cb9-c0ad486ec03c" volumeName="kubernetes.io/configmap/d09416ce-b095-4a28-9cb9-c0ad486ec03c-config" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861707    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1d895e08-f347-4b7c-b63e-26b06a3f6c10" volumeName="kubernetes.io/configmap/1d895e08-f347-4b7c-b63e-26b06a3f6c10-serviceca" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861720    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="870962af-eac0-484d-9267-db8ab42ef13d" volumeName="kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-image-registry-private-configuration" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861732    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2522cd43-48a3-40d4-9f8f-b7a0f778abbd" volumeName="kubernetes.io/projected/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-kube-api-access-gdkv4" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861762    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="417744b4-5ed7-4b20-9960-7147d2d567ee" volumeName="kubernetes.io/projected/417744b4-5ed7-4b20-9960-7147d2d567ee-kube-api-access-nlzhr" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861785    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d14f606-6852-4eff-b71f-5a8adcde5f84" volumeName="kubernetes.io/projected/4d14f606-6852-4eff-b71f-5a8adcde5f84-kube-api-access-v2c2n" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861798    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="698e630d-a5b0-43ce-9a12-aac9c60ab53c" volumeName="kubernetes.io/configmap/698e630d-a5b0-43ce-9a12-aac9c60ab53c-config" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861812    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="870962af-eac0-484d-9267-db8ab42ef13d" volumeName="kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-kube-api-access-9g6rr" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861825    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13" volumeName="kubernetes.io/projected/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-kube-api-access-nbn5k" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861838    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8ecf9a5-a6e5-460b-8cc9-44c996d57930" volumeName="kubernetes.io/configmap/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-service-ca-bundle" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861850    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef75aae9-bb5f-4e09-b374-fa398aadfae9" volumeName="kubernetes.io/projected/ef75aae9-bb5f-4e09-b374-fa398aadfae9-kube-api-access-mrt5f" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861862    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8ecf9a5-a6e5-460b-8cc9-44c996d57930" volumeName="kubernetes.io/empty-dir/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-snapshots" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861874    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b461004a-433c-4411-84df-b9aea9f682ab" volumeName="kubernetes.io/configmap/b461004a-433c-4411-84df-b9aea9f682ab-config-volume" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861887    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aa5b60a-ce95-4623-8ae2-32896ca1232d" volumeName="kubernetes.io/projected/0aa5b60a-ce95-4623-8ae2-32896ca1232d-kube-api-access-4sfr5" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861901    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2522cd43-48a3-40d4-9f8f-b7a0f778abbd" volumeName="kubernetes.io/empty-dir/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-tmp" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861915    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="698e630d-a5b0-43ce-9a12-aac9c60ab53c" volumeName="kubernetes.io/secret/698e630d-a5b0-43ce-9a12-aac9c60ab53c-serving-cert" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861927    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74e217e6-636d-4bab-b37f-c44dc0d8fe81" volumeName="kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-trusted-ca" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861945    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74e217e6-636d-4bab-b37f-c44dc0d8fe81" volumeName="kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-installation-pull-secrets" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861964    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13" volumeName="kubernetes.io/configmap/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-iptables-alerter-script" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861977    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8ecf9a5-a6e5-460b-8cc9-44c996d57930" volumeName="kubernetes.io/projected/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-kube-api-access-f68tp" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.861989    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c07ce523-d075-4203-be44-3f7d5af7e0d7" volumeName="kubernetes.io/projected/c07ce523-d075-4203-be44-3f7d5af7e0d7-kube-api-access-xv54m" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862001    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="417744b4-5ed7-4b20-9960-7147d2d567ee" volumeName="kubernetes.io/configmap/417744b4-5ed7-4b20-9960-7147d2d567ee-trusted-ca" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862016    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="870962af-eac0-484d-9267-db8ab42ef13d" volumeName="kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-bound-sa-token" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862029    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1926369-9621-4cac-b797-43a306bdb709" volumeName="kubernetes.io/configmap/b1926369-9621-4cac-b797-43a306bdb709-konnectivity-ca" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862042    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1926369-9621-4cac-b797-43a306bdb709" volumeName="kubernetes.io/secret/b1926369-9621-4cac-b797-43a306bdb709-agent-certs" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862056    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c73834fc-41c8-4eea-a739-52d16535890d" volumeName="kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862068    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc72c8e1-7e7c-4f53-946b-6ef94d6c5971" volumeName="kubernetes.io/projected/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-kube-api-access-mcz47" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862081    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc72c8e1-7e7c-4f53-946b-6ef94d6c5971" volumeName="kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-default-certificate" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862092    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eddb65a3-0fa8-48b5-8ec9-4c939e189be4" volumeName="kubernetes.io/projected/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-kube-api-access-rmpkm" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862116    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c13ee40-c34c-4575-8865-aceeb71759d0" volumeName="kubernetes.io/projected/1c13ee40-c34c-4575-8865-aceeb71759d0-kube-api-access-2c8q8" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862129    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b461004a-433c-4411-84df-b9aea9f682ab" volumeName="kubernetes.io/empty-dir/b461004a-433c-4411-84df-b9aea9f682ab-tmp-dir" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862143    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f3db2db6-6f0f-4af7-9bb7-493df9107bb9" volumeName="kubernetes.io/projected/f3db2db6-6f0f-4af7-9bb7-493df9107bb9-kube-api-access-2xrfz" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862155    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f3db2db6-6f0f-4af7-9bb7-493df9107bb9" volumeName="kubernetes.io/secret/f3db2db6-6f0f-4af7-9bb7-493df9107bb9-cert" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862167    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="213808a8-b7ec-4fc6-845b-d2648393d5bc" volumeName="kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862180    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30efbdba-8299-418b-b8f1-233c789443f5" volumeName="kubernetes.io/projected/30efbdba-8299-418b-b8f1-233c789443f5-kube-api-access-lxwj7" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862192    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="870962af-eac0-484d-9267-db8ab42ef13d" volumeName="kubernetes.io/empty-dir/870962af-eac0-484d-9267-db8ab42ef13d-ca-trust-extracted" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862204    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc72c8e1-7e7c-4f53-946b-6ef94d6c5971" volumeName="kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-stats-auth" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862218    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eddb65a3-0fa8-48b5-8ec9-4c939e189be4" volumeName="kubernetes.io/configmap/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-config" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862230    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1d895e08-f347-4b7c-b63e-26b06a3f6c10" volumeName="kubernetes.io/projected/1d895e08-f347-4b7c-b63e-26b06a3f6c10-kube-api-access-28wg4" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862242    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74e217e6-636d-4bab-b37f-c44dc0d8fe81" volumeName="kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-image-registry-private-configuration" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862255    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="789e64a1-501d-46d9-a58d-14f786410301" volumeName="kubernetes.io/projected/789e64a1-501d-46d9-a58d-14f786410301-kube-api-access-78j5f" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862267    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b2a74b29-e9ea-4ab9-b155-795d08684fe6" volumeName="kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-env-overrides" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862296    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b461004a-433c-4411-84df-b9aea9f682ab" volumeName="kubernetes.io/projected/b461004a-433c-4411-84df-b9aea9f682ab-kube-api-access-44d8q" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862312    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2522cd43-48a3-40d4-9f8f-b7a0f778abbd" volumeName="kubernetes.io/empty-dir/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-tuned" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862325    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="698e630d-a5b0-43ce-9a12-aac9c60ab53c" volumeName="kubernetes.io/projected/698e630d-a5b0-43ce-9a12-aac9c60ab53c-kube-api-access-fppqt" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862338    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74e217e6-636d-4bab-b37f-c44dc0d8fe81" volumeName="kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-kube-api-access-k9hwp" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862368    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30efbdba-8299-418b-b8f1-233c789443f5" volumeName="kubernetes.io/configmap/30efbdba-8299-418b-b8f1-233c789443f5-cni-binary-copy" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862382    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="417744b4-5ed7-4b20-9960-7147d2d567ee" volumeName="kubernetes.io/secret/417744b4-5ed7-4b20-9960-7147d2d567ee-serving-cert" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862395    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8ecf9a5-a6e5-460b-8cc9-44c996d57930" volumeName="kubernetes.io/configmap/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862408    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c63f4e36-582d-43be-8575-3b79b7c4756d" volumeName="kubernetes.io/configmap/c63f4e36-582d-43be-8575-3b79b7c4756d-nginx-conf" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862420    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c73834fc-41c8-4eea-a739-52d16535890d" volumeName="kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862432    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d09416ce-b095-4a28-9cb9-c0ad486ec03c" volumeName="kubernetes.io/projected/d09416ce-b095-4a28-9cb9-c0ad486ec03c-kube-api-access-kslbj" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862444    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff8251b4-1f23-4a86-9b1d-602c87cf133b" volumeName="kubernetes.io/projected/ff8251b4-1f23-4a86-9b1d-602c87cf133b-kube-api-access-885ds" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862464    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30efbdba-8299-418b-b8f1-233c789443f5" volumeName="kubernetes.io/configmap/30efbdba-8299-418b-b8f1-233c789443f5-multus-daemon-config" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862477    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74e217e6-636d-4bab-b37f-c44dc0d8fe81" volumeName="kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-certificates" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862490    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99ef12a9-a676-4258-a3a8-e7fcd611d158" volumeName="kubernetes.io/projected/99ef12a9-a676-4258-a3a8-e7fcd611d158-kube-api-access-k8qmb" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862502    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8ecf9a5-a6e5-460b-8cc9-44c996d57930" volumeName="kubernetes.io/empty-dir/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-tmp" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862516    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8ecf9a5-a6e5-460b-8cc9-44c996d57930" volumeName="kubernetes.io/secret/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-serving-cert" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862530    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b2a74b29-e9ea-4ab9-b155-795d08684fe6" volumeName="kubernetes.io/projected/b2a74b29-e9ea-4ab9-b155-795d08684fe6-kube-api-access-c8lmb" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862542    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c73834fc-41c8-4eea-a739-52d16535890d" volumeName="kubernetes.io/projected/c73834fc-41c8-4eea-a739-52d16535890d-kube-api-access-9tjn7" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862555    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d09416ce-b095-4a28-9cb9-c0ad486ec03c" volumeName="kubernetes.io/secret/d09416ce-b095-4a28-9cb9-c0ad486ec03c-serving-cert" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862567    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a994b38f-0096-4fff-b49c-e6d99dc5a6be" volumeName="kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862580    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff8251b4-1f23-4a86-9b1d-602c87cf133b" volumeName="kubernetes.io/empty-dir/ff8251b4-1f23-4a86-9b1d-602c87cf133b-tmp-dir" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862593    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74e217e6-636d-4bab-b37f-c44dc0d8fe81" volumeName="kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-bound-sa-token" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862606    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="870962af-eac0-484d-9267-db8ab42ef13d" volumeName="kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-trusted-ca" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862618    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="870962af-eac0-484d-9267-db8ab42ef13d" volumeName="kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-installation-pull-secrets" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862631    4865 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c73834fc-41c8-4eea-a739-52d16535890d" volumeName="kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-cni-binary-copy" seLinuxMountContext=""
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862643    4865 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 01:55:36.865938 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.862652    4865 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 01:55:36.877177 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.877149    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/1.log"
Nov 04 01:55:36.877335 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.877317    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/1.log"
Nov 04 01:55:36.886493 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.886453    4865 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 01:55:36.888019 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.887993    4865 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 01:55:36.888019 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.888019    4865 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 01:55:36.888182 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.888042    4865 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 01:55:36.888182 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:36.888087    4865 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 04 01:55:36.908955 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.908924    4865 generic.go:356] "Generic (PLEG): container finished" podID="c73834fc-41c8-4eea-a739-52d16535890d" containerID="8eb378ffb472b8adb4b21b30a7efe31c8601eb082671d06647339fa55d3d2dac" exitCode=0
Nov 04 01:55:36.908955 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.908951    4865 generic.go:356] "Generic (PLEG): container finished" podID="c73834fc-41c8-4eea-a739-52d16535890d" containerID="90c64767df1ba08401c4b652a3e18f845d6c7c8b98bd0e81a08fd0b65ccd1bbe" exitCode=0
Nov 04 01:55:36.908955 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.908960    4865 generic.go:356] "Generic (PLEG): container finished" podID="c73834fc-41c8-4eea-a739-52d16535890d" containerID="286be477883e536d2fd7ac3e03f29225f825bcbe340b5a4ccf70e28612b7c8b5" exitCode=0
Nov 04 01:55:36.909170 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.908969    4865 generic.go:356] "Generic (PLEG): container finished" podID="c73834fc-41c8-4eea-a739-52d16535890d" containerID="9441dbe372b048a5224e8a880cf3857f91e630836a9539a2b4ffcda697c43b1d" exitCode=0
Nov 04 01:55:36.909170 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.908978    4865 generic.go:356] "Generic (PLEG): container finished" podID="c73834fc-41c8-4eea-a739-52d16535890d" containerID="3d76081ad7dc212666e781b66847eb24565806762844d2c27e990f5ad43890cb" exitCode=0
Nov 04 01:55:36.909170 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.908984    4865 generic.go:356] "Generic (PLEG): container finished" podID="c73834fc-41c8-4eea-a739-52d16535890d" containerID="ca37d879af8640685efd83d07bcfc4c44d7d7482fc89c5f359d686e1d75d0115" exitCode=0
Nov 04 01:55:36.922484 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.922453    4865 generic.go:356] "Generic (PLEG): container finished" podID="f6b3c9ba0af4012b95306d2f2cfa3c61" containerID="a3f943c27344712b7b62303a8f6610281c8e8dcb8fa74233e6f62d57b55f567e" exitCode=0
Nov 04 01:55:36.932172 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.932145    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/1.log"
Nov 04 01:55:36.932329 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.932173    4865 generic.go:356] "Generic (PLEG): container finished" podID="417744b4-5ed7-4b20-9960-7147d2d567ee" containerID="5a240684c33f12cf1f818a2ddee27b424185731bb59eb98661d7f29c71edadc0" exitCode=255
Nov 04 01:55:36.945479 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.945458    4865 manager.go:324] Recovery completed
Nov 04 01:55:36.957391 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:36.957309    4865 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:55:36.964164 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.964145    4865 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:55:36.965383 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.965367    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:55:36.965457 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.965396    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:55:36.965457 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.965407    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:55:36.967453 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.967435    4865 cpu_manager.go:226] "Starting" policy="none"
Nov 04 01:55:36.967453 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.967450    4865 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 01:55:36.967602 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.967452    4865 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:55:36.967602 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.967474    4865 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 01:55:36.967708 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.967643    4865 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 04 01:55:36.967708 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.967659    4865 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 04 01:55:36.967874 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.967860    4865 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 04 01:55:36.967925 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.967876    4865 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 04 01:55:36.967925 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.967886    4865 policy_none.go:50] "Start"
Nov 04 01:55:36.967925 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.967898    4865 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 01:55:36.967925 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.967914    4865 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 01:55:36.968118 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.968104    4865 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 04 01:55:36.968169 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.968120    4865 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 04 01:55:36.968169 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.968128    4865 policy_none.go:44] "Start"
Nov 04 01:55:36.968412 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.968367    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:55:36.968412 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.968394    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:55:36.968412 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.968404    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:55:36.972396 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.972371    4865 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 01:55:36.974881 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.974868    4865 manager.go:341] "Starting Device Plugin manager"
Nov 04 01:55:36.974972 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:36.974893    4865 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 01:55:36.974972 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.974902    4865 server.go:87] "Starting device plugin registration server"
Nov 04 01:55:36.975207 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.975194    4865 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 01:55:36.975254 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.975210    4865 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 01:55:36.975319 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.975300    4865 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 01:55:36.975419 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.975393    4865 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 01:55:36.975465 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.975425    4865 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 01:55:36.975465 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.975432    4865 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 01:55:36.975559 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.975460    4865 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T01:55:36.975416973Z","UUID":"67cb9156-90bb-4eb2-a09a-2c0baa96cdff","Handler":null,"Name":"","Endpoint":""}
Nov 04 01:55:36.976659 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.976639    4865 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 01:55:36.976754 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.976677    4865 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 01:55:36.976754 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:36.976726    4865 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 01:55:36.976826 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:36.976759    4865 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-204.us-west-1.compute.internal\" not found"
Nov 04 01:55:36.988764 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.988319    4865 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal"]
Nov 04 01:55:36.989174 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.989093    4865 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:55:36.991703 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.991678    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:55:36.991703 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.991707    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:55:36.991822 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.991719    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:55:36.991998 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.991988    4865 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:55:36.992151 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.992135    4865 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:55:36.992706 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.992691    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:55:36.992706 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.992705    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:55:36.992830 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.992717    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:55:36.992830 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.992723    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:55:36.992830 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.992730    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:55:36.992830 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.992733    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:55:36.992937 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:36.992872    4865 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found" node="ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:36.993003 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.992976    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal" event={"ID":"56dfafb26cd80e2f2f640fddc6c0403a","Type":"ContainerStarted","Data":"5d7ccd269c59882bdea6d0caf7e4253bfc46abcb8bccb64fa7ba6046468ea164"}
Nov 04 01:55:36.993041 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.993010    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal" event={"ID":"56dfafb26cd80e2f2f640fddc6c0403a","Type":"ContainerStarted","Data":"a4cf7f99d75dbefad68a1a3ecfe1f4bc0aa617025acd7fe6bc3aa4552455a86a"}
Nov 04 01:55:36.993078 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.993041    4865 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:55:36.993078 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.993041    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal" event={"ID":"f6b3c9ba0af4012b95306d2f2cfa3c61","Type":"ContainerStarted","Data":"adaed33514693cd33b85c3aab17c6b59116cea6c71318536f65b4acc8c5e295f"}
Nov 04 01:55:36.993173 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.993078    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal" event={"ID":"f6b3c9ba0af4012b95306d2f2cfa3c61","Type":"ContainerDied","Data":"a3f943c27344712b7b62303a8f6610281c8e8dcb8fa74233e6f62d57b55f567e"}
Nov 04 01:55:36.993173 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.993091    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal" event={"ID":"f6b3c9ba0af4012b95306d2f2cfa3c61","Type":"ContainerStarted","Data":"c90dfcd721e4bf20cab1e57beb6127a78cb16c800014c20d17192be40af26ba5"}
Nov 04 01:55:36.993731 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.993706    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:55:36.993810 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.993736    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:55:36.993810 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:36.993751    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:55:36.993913 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:36.993903    4865 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-204.us-west-1.compute.internal\" not found" node="ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.058691 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.058647    4865 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:55:37.060194 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.060177    4865 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:55:37.076133 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.076116    4865 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:55:37.077180 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.077162    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:55:37.077295 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.077190    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:55:37.077295 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.077200    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:55:37.077295 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.077222    4865 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.109317 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.109273    4865 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:55:37.113206 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.113188    4865 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:55:37.114046 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.114024    4865 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 04 01:55:37.157345 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.157316    4865 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.212985 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.212954    4865 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.213185 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.213029    4865 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.213607 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.213583    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeNotReady"
Nov 04 01:55:37.213696 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.213628    4865 setters.go:546] "Node became not ready" node="ip-10-0-134-204.us-west-1.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-04T01:55:37Z","lastTransitionTime":"2026-11-04T01:55:37Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Nov 04 01:55:37.214842 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.214776    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6b3c9ba0af4012b95306d2f2cfa3c61-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"f6b3c9ba0af4012b95306d2f2cfa3c61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.214842 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.214803    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6b3c9ba0af4012b95306d2f2cfa3c61-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"f6b3c9ba0af4012b95306d2f2cfa3c61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.214842 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.214822    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/56dfafb26cd80e2f2f640fddc6c0403a-config\") pod \"kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"56dfafb26cd80e2f2f640fddc6c0403a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.222445 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.222392    4865 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:55:37.222445 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:37.222441    4865 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.222569 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.222456    4865 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.230067 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.230049    4865 nodeinfomanager.go:471] existing CSINode "ip-10-0-134-204.us-west-1.compute.internal" is owned by different node (oldNodeID="a886dcee-5669-4e84-b3c2-688303321273", newNodeID=""), cleaning up...
Nov 04 01:55:37.315710 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.315674    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6b3c9ba0af4012b95306d2f2cfa3c61-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"f6b3c9ba0af4012b95306d2f2cfa3c61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.315710 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.315712    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6b3c9ba0af4012b95306d2f2cfa3c61-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"f6b3c9ba0af4012b95306d2f2cfa3c61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.315908 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.315732    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/56dfafb26cd80e2f2f640fddc6c0403a-config\") pod \"kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"56dfafb26cd80e2f2f640fddc6c0403a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.315908 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.315778    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/56dfafb26cd80e2f2f640fddc6c0403a-config\") pod \"kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"56dfafb26cd80e2f2f640fddc6c0403a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.315908 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.315780    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6b3c9ba0af4012b95306d2f2cfa3c61-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"f6b3c9ba0af4012b95306d2f2cfa3c61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.315908 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.315781    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6b3c9ba0af4012b95306d2f2cfa3c61-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal\" (UID: \"f6b3c9ba0af4012b95306d2f2cfa3c61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.318978 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.318961    4865 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:55:37.319072 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:37.319008    4865 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.485513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.485430    4865 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-204.us-west-1.compute.internal" event="NodeReady"
Nov 04 01:55:37.485643 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.485523    4865 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 04 01:55:37.844029 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.843945    4865 apiserver.go:52] "Watching apiserver"
Nov 04 01:55:37.940808 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.940782    4865 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:37.940981 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:37.940889    4865 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:38.001318 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.001271    4865 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 01:55:38.001999 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.001968    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64","openshift-multus/multus-additional-cni-plugins-78wz6","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887","openshift-insights/insights-operator-7c4986d449-2bzkj","kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal","openshift-network-diagnostics/network-check-target-g6xmn","openshift-network-operator/iptables-alerter-2s59r","kube-system/kas-connection-checker-685895c7c9-rzbgn","kube-system/kas-connection-checker-685895c7c9-7gmmg","openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t","openshift-image-registry/image-registry-8574f7698d-qmq69","openshift-network-diagnostics/network-check-source-6f78d59f99-p2kf6","openshift-cluster-node-tuning-operator/tuned-h4glm","openshift-image-registry/image-registry-6c59b47685-9jmnr","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-cw5p4","openshift-multus/multus-n5mdk","openshift-network-console/networking-console-plugin-86c478bb88-8rvcg","openshift-insights/insights-runtime-extractor-fkhvv","openshift-dns/node-resolver-pj6wl","openshift-ovn-kubernetes/ovnkube-node-k76ph","openshift-dns/dns-default-68kbx","kube-system/kas-connection-checker-685895c7c9-p48wv","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p","openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz","kube-system/konnectivity-agent-dszxl","openshift-ingress-canary/ingress-canary-5ljw9","openshift-multus/network-metrics-daemon-74j82","kube-system/global-pull-secret-syncer-x48xl","openshift-console-operator/console-operator-7cd56b8dbc-jlqmg","openshift-image-registry/node-ca-ntp6j","openshift-ingress/router-default-56bdbb678c-mvkmq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"]
Nov 04 01:55:38.004992 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.004968    4865 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:55:38.005109 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.004991    4865 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:55:38.005109 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.005022    4865 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:38.005109 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.005028    4865 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-204.us-west-1.compute.internal"
Nov 04 01:55:38.005881 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.005860    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:38.006832 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.006812    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:38.008094 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.008077    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:38.008410 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.008395    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.008764 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.008751    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:38.009825 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.009809    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:38.015352 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.015336    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.016513 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.016496    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:38.017020 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.017004    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:38.059391 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.059371    4865 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 01:55:38.060363 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.060345    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 04 01:55:38.060433 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.060377    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 04 01:55:38.060433 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.060408    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 04 01:55:38.060559 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.060547    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 04 01:55:38.061832 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.061812    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 04 01:55:38.062707 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.062677    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.062774 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.062716    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.062981 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.062967    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 04 01:55:38.063035 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.062998    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 04 01:55:38.063035 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.063004    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.063035 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.063010    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.063035 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.063011    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 04 01:55:38.063211 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.063043    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.063211 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.063090    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.063211 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.063094    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.063211 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.063114    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.063211 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.063091    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.063211 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.063112    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.063211 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.063156    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.063211 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.063165    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.063211 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.063182    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.063211 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.063204    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.064206 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.064186    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.064323 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.064272    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 04 01:55:38.111478 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.111398    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 04 01:55:38.111478 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.111422    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.111478 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.111422    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 04 01:55:38.113724 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.113701    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.113724 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.113703    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.114803 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.114789    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 04 01:55:38.115271 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.115258    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 04 01:55:38.120084 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120053    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-service-ca-bundle\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:38.120167 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120098    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/30efbdba-8299-418b-b8f1-233c789443f5-multus-daemon-config\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.120167 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120115    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovnkube-config\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.120167 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120140    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff8251b4-1f23-4a86-9b1d-602c87cf133b-hosts-file\") pod \"node-resolver-pj6wl\" (UID: \"ff8251b4-1f23-4a86-9b1d-602c87cf133b\") " pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:55:38.120167 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120158    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-885ds\" (UniqueName: \"kubernetes.io/projected/ff8251b4-1f23-4a86-9b1d-602c87cf133b-kube-api-access-885ds\") pod \"node-resolver-pj6wl\" (UID: \"ff8251b4-1f23-4a86-9b1d-602c87cf133b\") " pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:55:38.120317 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120181    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-system-cni-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.120317 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120261    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d09416ce-b095-4a28-9cb9-c0ad486ec03c-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-84j4p\" (UID: \"d09416ce-b095-4a28-9cb9-c0ad486ec03c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"
Nov 04 01:55:38.120317 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120309    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-service-ca-bundle\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:38.120447 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120388    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d895e08-f347-4b7c-b63e-26b06a3f6c10-host\") pod \"node-ca-ntp6j\" (UID: \"1d895e08-f347-4b7c-b63e-26b06a3f6c10\") " pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:55:38.120496 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120486    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-trusted-ca\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.120549 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120508    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8trls\" (UniqueName: \"kubernetes.io/projected/1536e272-39b6-4702-a35d-0130eef3e290-kube-api-access-8trls\") pod \"network-metrics-daemon-74j82\" (UID: \"1536e272-39b6-4702-a35d-0130eef3e290\") " pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:38.120598 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120574    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d09416ce-b095-4a28-9cb9-c0ad486ec03c-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-84j4p\" (UID: \"d09416ce-b095-4a28-9cb9-c0ad486ec03c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"
Nov 04 01:55:38.120644 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120573    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-tuned\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.120697 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120683    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-tuned\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.120730 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120706    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-cni-binary-copy\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:55:38.120770 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120747    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrt5f\" (UniqueName: \"kubernetes.io/projected/ef75aae9-bb5f-4e09-b374-fa398aadfae9-kube-api-access-mrt5f\") pod \"kas-connection-checker-685895c7c9-p48wv\" (UID: \"ef75aae9-bb5f-4e09-b374-fa398aadfae9\") " pod="kube-system/kas-connection-checker-685895c7c9-p48wv"
Nov 04 01:55:38.120813 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120779    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv54m\" (UniqueName: \"kubernetes.io/projected/c07ce523-d075-4203-be44-3f7d5af7e0d7-kube-api-access-xv54m\") pod \"kas-connection-checker-685895c7c9-rzbgn\" (UID: \"c07ce523-d075-4203-be44-3f7d5af7e0d7\") " pod="kube-system/kas-connection-checker-685895c7c9-rzbgn"
Nov 04 01:55:38.120813 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120805    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-cni-bin\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.120907 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120827    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-cni-netd\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.120907 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120859    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/698e630d-a5b0-43ce-9a12-aac9c60ab53c-config\") pod \"service-ca-operator-544d749c8c-wzh4t\" (UID: \"698e630d-a5b0-43ce-9a12-aac9c60ab53c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"
Nov 04 01:55:38.120907 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120885    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-etc-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.121055 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120925    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-conf-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.121055 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120979    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-kubelet\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.121055 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.120998    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/40363d65-7ece-4f05-8668-439a60582a18-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:38.121055 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121034    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f68tp\" (UniqueName: \"kubernetes.io/projected/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-kube-api-access-f68tp\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:38.121222 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121080    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/698e630d-a5b0-43ce-9a12-aac9c60ab53c-config\") pod \"service-ca-operator-544d749c8c-wzh4t\" (UID: \"698e630d-a5b0-43ce-9a12-aac9c60ab53c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"
Nov 04 01:55:38.121222 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121133    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1926369-9621-4cac-b797-43a306bdb709-agent-certs\") pod \"konnectivity-agent-dszxl\" (UID: \"b1926369-9621-4cac-b797-43a306bdb709\") " pod="kube-system/konnectivity-agent-dszxl"
Nov 04 01:55:38.121222 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121210    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/40363d65-7ece-4f05-8668-439a60582a18-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:38.121377 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121230    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/417744b4-5ed7-4b20-9960-7147d2d567ee-config\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:38.121377 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121253    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2c2n\" (UniqueName: \"kubernetes.io/projected/4d14f606-6852-4eff-b71f-5a8adcde5f84-kube-api-access-v2c2n\") pod \"network-check-source-6f78d59f99-p2kf6\" (UID: \"4d14f606-6852-4eff-b71f-5a8adcde5f84\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-p2kf6"
Nov 04 01:55:38.121377 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121270    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-lib-modules\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.121463 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121405    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/417744b4-5ed7-4b20-9960-7147d2d567ee-config\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:38.121463 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121446    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-cni-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.121523 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121468    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxwj7\" (UniqueName: \"kubernetes.io/projected/30efbdba-8299-418b-b8f1-233c789443f5-kube-api-access-lxwj7\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.121523 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121490    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-installation-pull-secrets\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.121523 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121511    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-tmp\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:38.121621 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121532    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-default-certificate\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:38.121621 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121548    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:38.121621 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121562    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-env-overrides\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.121621 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121586    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28wg4\" (UniqueName: \"kubernetes.io/projected/1d895e08-f347-4b7c-b63e-26b06a3f6c10-kube-api-access-28wg4\") pod \"node-ca-ntp6j\" (UID: \"1d895e08-f347-4b7c-b63e-26b06a3f6c10\") " pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:55:38.121621 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121617    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysconfig\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.121836 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121639    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-host\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.121836 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121640    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-tmp\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:38.121836 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121677    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b655g\" (UniqueName: \"kubernetes.io/projected/40363d65-7ece-4f05-8668-439a60582a18-kube-api-access-b655g\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:38.121836 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121695    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:55:38.121836 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121717    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56z2g\" (UniqueName: \"kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g\") pod \"network-check-target-g6xmn\" (UID: \"213808a8-b7ec-4fc6-845b-d2648393d5bc\") " pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:38.121836 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121737    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-stats-auth\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:38.121836 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121772    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-run-ovn-kubernetes\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.121836 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121788    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-run-netns\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.121836 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121806    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-sys\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.121836 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121801    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:38.122143 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121849    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdkv4\" (UniqueName: \"kubernetes.io/projected/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-kube-api-access-gdkv4\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.122143 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.121980    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:38.122143 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122006    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-node-log\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.122143 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122021    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-log-socket\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.122143 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122040    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovn-node-metrics-cert\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.122143 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122074    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/417744b4-5ed7-4b20-9960-7147d2d567ee-serving-cert\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:38.122326 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122143    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:55:38.122326 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122176    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-serving-cert\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:38.122326 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122203    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-netns\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.122326 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122231    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3db2db6-6f0f-4af7-9bb7-493df9107bb9-cert\") pod \"ingress-canary-5ljw9\" (UID: \"f3db2db6-6f0f-4af7-9bb7-493df9107bb9\") " pod="openshift-ingress-canary/ingress-canary-5ljw9"
Nov 04 01:55:38.122326 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122256    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlzhr\" (UniqueName: \"kubernetes.io/projected/417744b4-5ed7-4b20-9960-7147d2d567ee-kube-api-access-nlzhr\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:38.122326 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122274    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-iptables-alerter-script\") pod \"iptables-alerter-2s59r\" (UID: \"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13\") " pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:55:38.122326 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122320    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-system-cni-dir\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:55:38.122552 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122321    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/417744b4-5ed7-4b20-9960-7147d2d567ee-serving-cert\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:38.122552 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122354    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a994b38f-0096-4fff-b49c-e6d99dc5a6be-kubelet-config\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:38.122552 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122382    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c63f4e36-582d-43be-8575-3b79b7c4756d-nginx-conf\") pod \"networking-console-plugin-86c478bb88-8rvcg\" (UID: \"c63f4e36-582d-43be-8575-3b79b7c4756d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:38.122552 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122414    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/789e64a1-501d-46d9-a58d-14f786410301-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:38.122552 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122435    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-cnibin\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.122552 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122464    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:38.122552 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122493    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d09416ce-b095-4a28-9cb9-c0ad486ec03c-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-84j4p\" (UID: \"d09416ce-b095-4a28-9cb9-c0ad486ec03c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"
Nov 04 01:55:38.122552 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122528    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-bound-sa-token\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.122773 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122594    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9hwp\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-kube-api-access-k9hwp\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.122773 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122618    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d09416ce-b095-4a28-9cb9-c0ad486ec03c-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-84j4p\" (UID: \"d09416ce-b095-4a28-9cb9-c0ad486ec03c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"
Nov 04 01:55:38.122773 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122642    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-cni-multus\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.122773 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122668    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovnkube-script-lib\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.122773 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122685    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-registry-certificates\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.122773 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122714    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g6rr\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-kube-api-access-9g6rr\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.122773 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122736    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-sys-fs\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.122773 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122753    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sfr5\" (UniqueName: \"kubernetes.io/projected/0aa5b60a-ce95-4623-8ae2-32896ca1232d-kube-api-access-4sfr5\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.123055 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122775    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-image-registry-private-configuration\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.123055 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122796    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcz47\" (UniqueName: \"kubernetes.io/projected/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-kube-api-access-mcz47\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:38.123055 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122811    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1d895e08-f347-4b7c-b63e-26b06a3f6c10-serviceca\") pod \"node-ca-ntp6j\" (UID: \"1d895e08-f347-4b7c-b63e-26b06a3f6c10\") " pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:55:38.123055 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122830    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xrfz\" (UniqueName: \"kubernetes.io/projected/f3db2db6-6f0f-4af7-9bb7-493df9107bb9-kube-api-access-2xrfz\") pod \"ingress-canary-5ljw9\" (UID: \"f3db2db6-6f0f-4af7-9bb7-493df9107bb9\") " pod="openshift-ingress-canary/ingress-canary-5ljw9"
Nov 04 01:55:38.123055 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122853    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:38.123055 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122902    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-var-lib-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.123055 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122927    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-cnibin\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:55:38.123055 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122945    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.123055 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.122982    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.123055 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123005    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-var-lib-kubelet\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.123055 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123026    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.123055 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123045    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/417744b4-5ed7-4b20-9960-7147d2d567ee-trusted-ca\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:38.123599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123073    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysctl-conf\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.123599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123099    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-8rvcg\" (UID: \"c63f4e36-582d-43be-8575-3b79b7c4756d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:38.123599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123152    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-installation-pull-secrets\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.123599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123211    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:38.123599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123258    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-systemd\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.123599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123307    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-ovn\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.123599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123321    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/417744b4-5ed7-4b20-9960-7147d2d567ee-trusted-ca\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:38.123599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123384    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-etc-selinux\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.123599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123404    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1926369-9621-4cac-b797-43a306bdb709-konnectivity-ca\") pod \"konnectivity-agent-dszxl\" (UID: \"b1926369-9621-4cac-b797-43a306bdb709\") " pod="kube-system/konnectivity-agent-dszxl"
Nov 04 01:55:38.123599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123423    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbn5k\" (UniqueName: \"kubernetes.io/projected/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-kube-api-access-nbn5k\") pod \"iptables-alerter-2s59r\" (UID: \"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13\") " pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:55:38.123599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123445    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-systemd-units\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.123599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123459    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-socket-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.123599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123478    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:55:38.123599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123514    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-socket-dir-parent\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.123599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123559    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:38.123599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123585    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-image-registry-private-configuration\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.123599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123587    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123642    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-bound-sa-token\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123683    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-kubernetes\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123718    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b461004a-433c-4411-84df-b9aea9f682ab-tmp-dir\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123767    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44d8q\" (UniqueName: \"kubernetes.io/projected/b461004a-433c-4411-84df-b9aea9f682ab-kube-api-access-44d8q\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123816    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78j5f\" (UniqueName: \"kubernetes.io/projected/789e64a1-501d-46d9-a58d-14f786410301-kube-api-access-78j5f\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123846    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-k8s-cni-cncf-io\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123856    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b461004a-433c-4411-84df-b9aea9f682ab-tmp-dir\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123871    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-systemd\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123897    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123917    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-kubelet\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123933    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-etc-kubernetes\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.123952    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c8q8\" (UniqueName: \"kubernetes.io/projected/1c13ee40-c34c-4575-8865-aceeb71759d0-kube-api-access-2c8q8\") pod \"volume-data-source-validator-58c9f5f46d-cw5p4\" (UID: \"1c13ee40-c34c-4575-8865-aceeb71759d0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-cw5p4"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124062    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-slash\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124087    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124109    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a994b38f-0096-4fff-b49c-e6d99dc5a6be-dbus\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124128    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124204    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-host-slash\") pod \"iptables-alerter-2s59r\" (UID: \"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13\") " pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124237    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b461004a-433c-4411-84df-b9aea9f682ab-config-volume\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124254    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30efbdba-8299-418b-b8f1-233c789443f5-cni-binary-copy\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.124325 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124272    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-cni-bin\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124369    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-modprobe-d\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124401    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-certificates\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124423    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-snapshots\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124445    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/698e630d-a5b0-43ce-9a12-aac9c60ab53c-serving-cert\") pod \"service-ca-operator-544d749c8c-wzh4t\" (UID: \"698e630d-a5b0-43ce-9a12-aac9c60ab53c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124462    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/789e64a1-501d-46d9-a58d-14f786410301-crio-socket\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124476    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/789e64a1-501d-46d9-a58d-14f786410301-data-volume\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124498    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8lmb\" (UniqueName: \"kubernetes.io/projected/b2a74b29-e9ea-4ab9-b155-795d08684fe6-kube-api-access-c8lmb\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124524    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs\") pod \"network-metrics-daemon-74j82\" (UID: \"1536e272-39b6-4702-a35d-0130eef3e290\") " pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124542    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/789e64a1-501d-46d9-a58d-14f786410301-data-volume\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124550    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmpkm\" (UniqueName: \"kubernetes.io/projected/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-kube-api-access-rmpkm\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124571    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-snapshots\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124579    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/74e217e6-636d-4bab-b37f-c44dc0d8fe81-ca-trust-extracted\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124615    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-device-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124629    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/74e217e6-636d-4bab-b37f-c44dc0d8fe81-ca-trust-extracted\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124620    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/698e630d-a5b0-43ce-9a12-aac9c60ab53c-serving-cert\") pod \"service-ca-operator-544d749c8c-wzh4t\" (UID: \"698e630d-a5b0-43ce-9a12-aac9c60ab53c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124645    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysctl-d\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124690    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff8251b4-1f23-4a86-9b1d-602c87cf133b-tmp-dir\") pod \"node-resolver-pj6wl\" (UID: \"ff8251b4-1f23-4a86-9b1d-602c87cf133b\") " pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124717    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124750    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff8251b4-1f23-4a86-9b1d-602c87cf133b-tmp-dir\") pod \"node-resolver-pj6wl\" (UID: \"ff8251b4-1f23-4a86-9b1d-602c87cf133b\") " pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124753    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fppqt\" (UniqueName: \"kubernetes.io/projected/698e630d-a5b0-43ce-9a12-aac9c60ab53c-kube-api-access-fppqt\") pod \"service-ca-operator-544d749c8c-wzh4t\" (UID: \"698e630d-a5b0-43ce-9a12-aac9c60ab53c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124784    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-multus-certs\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124815    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-tmp\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124844    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124869    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-registration-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124894    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tjn7\" (UniqueName: \"kubernetes.io/projected/c73834fc-41c8-4eea-a739-52d16535890d-kube-api-access-9tjn7\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124909    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-tmp\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.124988    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-hostroot\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.125012    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kslbj\" (UniqueName: \"kubernetes.io/projected/d09416ce-b095-4a28-9cb9-c0ad486ec03c-kube-api-access-kslbj\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-84j4p\" (UID: \"d09416ce-b095-4a28-9cb9-c0ad486ec03c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"
Nov 04 01:55:38.125048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.125046    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-run\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.125829 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.125080    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-trusted-ca\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.125829 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.125115    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8qmb\" (UniqueName: \"kubernetes.io/projected/99ef12a9-a676-4258-a3a8-e7fcd611d158-kube-api-access-k8qmb\") pod \"kas-connection-checker-685895c7c9-7gmmg\" (UID: \"99ef12a9-a676-4258-a3a8-e7fcd611d158\") " pod="kube-system/kas-connection-checker-685895c7c9-7gmmg"
Nov 04 01:55:38.125829 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.125178    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/870962af-eac0-484d-9267-db8ab42ef13d-ca-trust-extracted\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.125829 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.125275    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/870962af-eac0-484d-9267-db8ab42ef13d-ca-trust-extracted\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.161014 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.160987    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 01:55:38.161143 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.160984    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 01:55:38.161143 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161038    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 04 01:55:38.161143 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161063    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 01:55:38.161263 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161064    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 04 01:55:38.161263 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161088    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 04 01:55:38.161372 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161357    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7qkcb\""
Nov 04 01:55:38.161522 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161507    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 04 01:55:38.161578 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161527    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 04 01:55:38.161578 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161538    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 04 01:55:38.161770 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161585    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 04 01:55:38.161770 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161764    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bxxxw\""
Nov 04 01:55:38.161933 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161744    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tvbnh\""
Nov 04 01:55:38.161933 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161759    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 01:55:38.161933 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161835    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 01:55:38.161933 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161870    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-default-certificate\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:38.161933 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161891    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zsfkj\""
Nov 04 01:55:38.161933 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161929    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-9ldb4\""
Nov 04 01:55:38.162226 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161937    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-tlvkz\""
Nov 04 01:55:38.162226 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.161939    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 01:55:38.162226 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162033    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-stats-auth\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:38.162226 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162057    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k89rs\""
Nov 04 01:55:38.162226 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162095    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zzzll\""
Nov 04 01:55:38.162226 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162104    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nrwjt\""
Nov 04 01:55:38.162226 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162105    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-qmgtz\""
Nov 04 01:55:38.162226 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162112    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-zk98g\""
Nov 04 01:55:38.162226 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162122    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zhvgj\""
Nov 04 01:55:38.162226 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162109    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mj9m5\""
Nov 04 01:55:38.162226 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162144    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r7nnn\""
Nov 04 01:55:38.162226 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162157    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-s4jjj\""
Nov 04 01:55:38.162226 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162165    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bzrbt\""
Nov 04 01:55:38.162226 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162180    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 01:55:38.162226 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162186    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 04 01:55:38.162226 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162230    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 01:55:38.163048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162302    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-wpjt7\""
Nov 04 01:55:38.163048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162316    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovn-node-metrics-cert\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.163048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162367    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-serving-cert\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:38.163048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162456    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3db2db6-6f0f-4af7-9bb7-493df9107bb9-cert\") pod \"ingress-canary-5ljw9\" (UID: \"f3db2db6-6f0f-4af7-9bb7-493df9107bb9\") " pod="openshift-ingress-canary/ingress-canary-5ljw9"
Nov 04 01:55:38.163048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162815    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.163048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162827    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.163048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162839    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.163048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162858    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.163048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.162982    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-image-registry-private-configuration\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.163048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163037    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 04 01:55:38.163048 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163048    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.163580 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163096    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.163580 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163144    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 01:55:38.163580 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163342    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 01:55:38.163580 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163404    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-installation-pull-secrets\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.163580 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163414    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.163580 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163454    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.163580 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163470    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 01:55:38.163580 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163480    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.163580 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163485    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.163580 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163537    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 01:55:38.164022 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163648    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.164022 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163658    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 04 01:55:38.164022 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163664    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.164022 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163669    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 01:55:38.164022 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163667    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.164022 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163698    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.164022 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163717    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 01:55:38.164022 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163705    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.164022 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163734    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 01:55:38.164022 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163741    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 01:55:38.164022 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163752    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.164022 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163775    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 01:55:38.164022 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163843    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-image-registry-private-configuration\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.164022 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163872    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 01:55:38.164022 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163905    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 01:55:38.164022 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.163912    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 01:55:38.164652 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.164122    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 01:55:38.164652 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.164167    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 01:55:38.164652 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.164437    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a994b38f-0096-4fff-b49c-e6d99dc5a6be-original-pull-secret\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:38.164652 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.164444    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:55:38.164652 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.164520    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30efbdba-8299-418b-b8f1-233c789443f5-cni-binary-copy\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.164652 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.164549    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-certificates\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.164652 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.164577    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b461004a-433c-4411-84df-b9aea9f682ab-config-volume\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:38.164843 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.164671    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 04 01:55:38.166186 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.166168    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-trusted-ca\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.170994 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.170975    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovnkube-config\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.170994 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.170987    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/30efbdba-8299-418b-b8f1-233c789443f5-multus-daemon-config\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.171061 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.171026    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-cni-binary-copy\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:55:38.171061 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.171026    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-trusted-ca\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.172166 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.172151    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-env-overrides\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.172206 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.172176    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-installation-pull-secrets\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.172314 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.172298    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1926369-9621-4cac-b797-43a306bdb709-agent-certs\") pod \"konnectivity-agent-dszxl\" (UID: \"b1926369-9621-4cac-b797-43a306bdb709\") " pod="kube-system/konnectivity-agent-dszxl"
Nov 04 01:55:38.172465 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.172448    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c73834fc-41c8-4eea-a739-52d16535890d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:55:38.172626 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.172609    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-iptables-alerter-script\") pod \"iptables-alerter-2s59r\" (UID: \"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13\") " pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:55:38.172663 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.172612    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c63f4e36-582d-43be-8575-3b79b7c4756d-nginx-conf\") pod \"networking-console-plugin-86c478bb88-8rvcg\" (UID: \"c63f4e36-582d-43be-8575-3b79b7c4756d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:38.173330 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.173314    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-registry-certificates\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.173364 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.173349    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b2a74b29-e9ea-4ab9-b155-795d08684fe6-ovnkube-script-lib\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.173443 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.173428    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1d895e08-f347-4b7c-b63e-26b06a3f6c10-serviceca\") pod \"node-ca-ntp6j\" (UID: \"1d895e08-f347-4b7c-b63e-26b06a3f6c10\") " pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:55:38.173833 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.173817    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1926369-9621-4cac-b797-43a306bdb709-konnectivity-ca\") pod \"konnectivity-agent-dszxl\" (UID: \"b1926369-9621-4cac-b797-43a306bdb709\") " pod="kube-system/konnectivity-agent-dszxl"
Nov 04 01:55:38.181594 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.181577    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrt5f\" (UniqueName: \"kubernetes.io/projected/ef75aae9-bb5f-4e09-b374-fa398aadfae9-kube-api-access-mrt5f\") pod \"kas-connection-checker-685895c7c9-p48wv\" (UID: \"ef75aae9-bb5f-4e09-b374-fa398aadfae9\") " pod="kube-system/kas-connection-checker-685895c7c9-p48wv"
Nov 04 01:55:38.181732 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.181717    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8trls\" (UniqueName: \"kubernetes.io/projected/1536e272-39b6-4702-a35d-0130eef3e290-kube-api-access-8trls\") pod \"network-metrics-daemon-74j82\" (UID: \"1536e272-39b6-4702-a35d-0130eef3e290\") " pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:38.182527 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.182510    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxwj7\" (UniqueName: \"kubernetes.io/projected/30efbdba-8299-418b-b8f1-233c789443f5-kube-api-access-lxwj7\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.182653 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.182631    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b655g\" (UniqueName: \"kubernetes.io/projected/40363d65-7ece-4f05-8668-439a60582a18-kube-api-access-b655g\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:38.185024 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.185007    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-885ds\" (UniqueName: \"kubernetes.io/projected/ff8251b4-1f23-4a86-9b1d-602c87cf133b-kube-api-access-885ds\") pod \"node-resolver-pj6wl\" (UID: \"ff8251b4-1f23-4a86-9b1d-602c87cf133b\") " pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:55:38.185215 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.185200    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28wg4\" (UniqueName: \"kubernetes.io/projected/1d895e08-f347-4b7c-b63e-26b06a3f6c10-kube-api-access-28wg4\") pod \"node-ca-ntp6j\" (UID: \"1d895e08-f347-4b7c-b63e-26b06a3f6c10\") " pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:55:38.185371 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.185355    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2c2n\" (UniqueName: \"kubernetes.io/projected/4d14f606-6852-4eff-b71f-5a8adcde5f84-kube-api-access-v2c2n\") pod \"network-check-source-6f78d59f99-p2kf6\" (UID: \"4d14f606-6852-4eff-b71f-5a8adcde5f84\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-p2kf6"
Nov 04 01:55:38.186645 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.186624    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv54m\" (UniqueName: \"kubernetes.io/projected/c07ce523-d075-4203-be44-3f7d5af7e0d7-kube-api-access-xv54m\") pod \"kas-connection-checker-685895c7c9-rzbgn\" (UID: \"c07ce523-d075-4203-be44-3f7d5af7e0d7\") " pod="kube-system/kas-connection-checker-685895c7c9-rzbgn"
Nov 04 01:55:38.187433 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.187417    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f68tp\" (UniqueName: \"kubernetes.io/projected/a8ecf9a5-a6e5-460b-8cc9-44c996d57930-kube-api-access-f68tp\") pod \"insights-operator-7c4986d449-2bzkj\" (UID: \"a8ecf9a5-a6e5-460b-8cc9-44c996d57930\") " pod="openshift-insights/insights-operator-7c4986d449-2bzkj"
Nov 04 01:55:38.193192 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.193167    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44d8q\" (UniqueName: \"kubernetes.io/projected/b461004a-433c-4411-84df-b9aea9f682ab-kube-api-access-44d8q\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:38.193317 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.193194    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sfr5\" (UniqueName: \"kubernetes.io/projected/0aa5b60a-ce95-4623-8ae2-32896ca1232d-kube-api-access-4sfr5\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.193317 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.193309    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fppqt\" (UniqueName: \"kubernetes.io/projected/698e630d-a5b0-43ce-9a12-aac9c60ab53c-kube-api-access-fppqt\") pod \"service-ca-operator-544d749c8c-wzh4t\" (UID: \"698e630d-a5b0-43ce-9a12-aac9c60ab53c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t"
Nov 04 01:55:38.193484 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.193465    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9hwp\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-kube-api-access-k9hwp\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.197647 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.194729    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-bound-sa-token\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.197647 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.194751    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tjn7\" (UniqueName: \"kubernetes.io/projected/c73834fc-41c8-4eea-a739-52d16535890d-kube-api-access-9tjn7\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:55:38.197647 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.194891    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xrfz\" (UniqueName: \"kubernetes.io/projected/f3db2db6-6f0f-4af7-9bb7-493df9107bb9-kube-api-access-2xrfz\") pod \"ingress-canary-5ljw9\" (UID: \"f3db2db6-6f0f-4af7-9bb7-493df9107bb9\") " pod="openshift-ingress-canary/ingress-canary-5ljw9"
Nov 04 01:55:38.197647 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.194993    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c8q8\" (UniqueName: \"kubernetes.io/projected/1c13ee40-c34c-4575-8865-aceeb71759d0-kube-api-access-2c8q8\") pod \"volume-data-source-validator-58c9f5f46d-cw5p4\" (UID: \"1c13ee40-c34c-4575-8865-aceeb71759d0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-cw5p4"
Nov 04 01:55:38.197647 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.195305    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmpkm\" (UniqueName: \"kubernetes.io/projected/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-kube-api-access-rmpkm\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:38.197940 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.197811    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-bound-sa-token\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.198122 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.198102    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbn5k\" (UniqueName: \"kubernetes.io/projected/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-kube-api-access-nbn5k\") pod \"iptables-alerter-2s59r\" (UID: \"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13\") " pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:55:38.198475 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.198442    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8lmb\" (UniqueName: \"kubernetes.io/projected/b2a74b29-e9ea-4ab9-b155-795d08684fe6-kube-api-access-c8lmb\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.198564 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.198452    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdkv4\" (UniqueName: \"kubernetes.io/projected/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-kube-api-access-gdkv4\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.199625 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.199149    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8qmb\" (UniqueName: \"kubernetes.io/projected/99ef12a9-a676-4258-a3a8-e7fcd611d158-kube-api-access-k8qmb\") pod \"kas-connection-checker-685895c7c9-7gmmg\" (UID: \"99ef12a9-a676-4258-a3a8-e7fcd611d158\") " pod="kube-system/kas-connection-checker-685895c7c9-7gmmg"
Nov 04 01:55:38.199625 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.199254    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcz47\" (UniqueName: \"kubernetes.io/projected/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-kube-api-access-mcz47\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:38.199625 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.199474    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kslbj\" (UniqueName: \"kubernetes.io/projected/d09416ce-b095-4a28-9cb9-c0ad486ec03c-kube-api-access-kslbj\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-84j4p\" (UID: \"d09416ce-b095-4a28-9cb9-c0ad486ec03c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p"
Nov 04 01:55:38.199625 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.199590    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78j5f\" (UniqueName: \"kubernetes.io/projected/789e64a1-501d-46d9-a58d-14f786410301-kube-api-access-78j5f\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:38.199625 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.199591    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlzhr\" (UniqueName: \"kubernetes.io/projected/417744b4-5ed7-4b20-9960-7147d2d567ee-kube-api-access-nlzhr\") pod \"console-operator-7cd56b8dbc-jlqmg\" (UID: \"417744b4-5ed7-4b20-9960-7147d2d567ee\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:38.199819 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.199656    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56z2g\" (UniqueName: \"kubernetes.io/projected/213808a8-b7ec-4fc6-845b-d2648393d5bc-kube-api-access-56z2g\") pod \"network-check-target-g6xmn\" (UID: \"213808a8-b7ec-4fc6-845b-d2648393d5bc\") " pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:38.199819 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.199666    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g6rr\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-kube-api-access-9g6rr\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.225563 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225528    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysctl-conf\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.225563 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225568    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-8rvcg\" (UID: \"c63f4e36-582d-43be-8575-3b79b7c4756d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:38.225700 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225585    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-systemd\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.225700 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225600    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-ovn\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.225700 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225615    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-etc-selinux\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.225700 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225638    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-systemd-units\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.225700 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225655    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-systemd\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.225700 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225662    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-socket-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.225700 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225684    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-ovn\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.225700 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225696    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysctl-conf\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.225961 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.225709    4865 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:55:38.225961 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225740    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-systemd-units\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.225961 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225709    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-etc-selinux\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.225961 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225743    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-socket-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.225961 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225759    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-socket-dir-parent\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.225961 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225710    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-socket-dir-parent\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.225961 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.225817    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert podName:c63f4e36-582d-43be-8575-3b79b7c4756d nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.725802205 +0000 UTC m=+1.952588866 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-8rvcg" (UID: "c63f4e36-582d-43be-8575-3b79b7c4756d") : secret "networking-console-plugin-cert" not found
Nov 04 01:55:38.225961 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225834    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:38.225961 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225867    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-kubernetes\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.225961 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225892    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-k8s-cni-cncf-io\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.225961 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225912    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-systemd\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.225961 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.225917    4865 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:55:38.225961 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225934    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.225961 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225956    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-kubelet\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.225961 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225964    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-k8s-cni-cncf-io\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.225981    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls podName:eddb65a3-0fa8-48b5-8ec9-4c939e189be4 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.725960992 +0000 UTC m=+1.952747655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-sr887" (UID: "eddb65a3-0fa8-48b5-8ec9-4c939e189be4") : secret "samples-operator-tls" not found
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225991    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-kubernetes\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225993    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-kubelet\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226008    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-etc-kubernetes\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.226028    4865 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226034    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-slash\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.225992    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-systemd\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226057    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-etc-kubernetes\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.226040    4865 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8574f7698d-qmq69: secret "image-registry-tls" not found
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226081    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-slash\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.226116    4865 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226063    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.226147    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls podName:74e217e6-636d-4bab-b37f-c44dc0d8fe81 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.726132162 +0000 UTC m=+1.952918827 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls") pod "image-registry-8574f7698d-qmq69" (UID: "74e217e6-636d-4bab-b37f-c44dc0d8fe81") : secret "image-registry-tls" not found
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.226183    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls podName:b461004a-433c-4411-84df-b9aea9f682ab nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.72617276 +0000 UTC m=+1.952959408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls") pod "dns-default-68kbx" (UID: "b461004a-433c-4411-84df-b9aea9f682ab") : secret "dns-default-metrics-tls" not found
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226194    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a994b38f-0096-4fff-b49c-e6d99dc5a6be-dbus\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226209    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-host-slash\") pod \"iptables-alerter-2s59r\" (UID: \"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13\") " pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226225    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-cni-bin\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226242    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a994b38f-0096-4fff-b49c-e6d99dc5a6be-dbus\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226242    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-modprobe-d\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226266    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13-host-slash\") pod \"iptables-alerter-2s59r\" (UID: \"a3f2a882-c7a7-4227-9c3e-bbfb5fbadf13\") " pod="openshift-network-operator/iptables-alerter-2s59r"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226273    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-cni-bin\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226274    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/789e64a1-501d-46d9-a58d-14f786410301-crio-socket\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226320    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/789e64a1-501d-46d9-a58d-14f786410301-crio-socket\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226326    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs\") pod \"network-metrics-daemon-74j82\" (UID: \"1536e272-39b6-4702-a35d-0130eef3e290\") " pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226351    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-device-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226357    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-modprobe-d\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226374    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysctl-d\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226394    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226416    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-multus-certs\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226434    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-device-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.226441    4865 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226453    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-multus-certs\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.226486    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs podName:1536e272-39b6-4702-a35d-0130eef3e290 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.726476727 +0000 UTC m=+1.953263394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs") pod "network-metrics-daemon-74j82" (UID: "1536e272-39b6-4702-a35d-0130eef3e290") : secret "metrics-daemon-secret" not found
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226495    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysctl-d\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.226515    4865 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.226545    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls podName:40363d65-7ece-4f05-8668-439a60582a18 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.726536592 +0000 UTC m=+1.953323251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wzmdz" (UID: "40363d65-7ece-4f05-8668-439a60582a18") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:38.226539 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226560    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226576    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-registration-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226591    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-hostroot\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226599    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226605    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-run\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226625    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff8251b4-1f23-4a86-9b1d-602c87cf133b-hosts-file\") pod \"node-resolver-pj6wl\" (UID: \"ff8251b4-1f23-4a86-9b1d-602c87cf133b\") " pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226633    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-registration-dir\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226645    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-system-cni-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226661    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-run\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226663    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff8251b4-1f23-4a86-9b1d-602c87cf133b-hosts-file\") pod \"node-resolver-pj6wl\" (UID: \"ff8251b4-1f23-4a86-9b1d-602c87cf133b\") " pod="openshift-dns/node-resolver-pj6wl"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226664    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d895e08-f347-4b7c-b63e-26b06a3f6c10-host\") pod \"node-ca-ntp6j\" (UID: \"1d895e08-f347-4b7c-b63e-26b06a3f6c10\") " pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226689    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-system-cni-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226690    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d895e08-f347-4b7c-b63e-26b06a3f6c10-host\") pod \"node-ca-ntp6j\" (UID: \"1d895e08-f347-4b7c-b63e-26b06a3f6c10\") " pod="openshift-image-registry/node-ca-ntp6j"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226699    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-cni-bin\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226662    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-hostroot\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226718    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-cni-netd\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226720    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-cni-bin\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226734    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-etc-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226748    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-conf-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226762    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-kubelet\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226763    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-cni-netd\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226775    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-etc-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226778    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-lib-modules\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226807    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-cni-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226817    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-kubelet\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226809    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-conf-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226838    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysconfig\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226850    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-lib-modules\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226857    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-host\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226865    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-multus-cni-dir\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226874    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226891    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-run-ovn-kubernetes\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226902    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-host\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226892    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-etc-sysconfig\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226915    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-run-netns\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226938    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-run-netns\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226939    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-sys\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226937    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-run-ovn-kubernetes\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226962    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226977    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-node-log\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226920    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226989    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-sys\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.226991    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-log-socket\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227027    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-node-log\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.227033    4865 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227036    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-netns\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227065    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-system-cni-dir\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.227071    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls podName:789e64a1-501d-46d9-a58d-14f786410301 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.727060706 +0000 UTC m=+1.953847352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkhvv" (UID: "789e64a1-501d-46d9-a58d-14f786410301") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227089    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-log-socket\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227096    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a994b38f-0096-4fff-b49c-e6d99dc5a6be-kubelet-config\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227124    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/789e64a1-501d-46d9-a58d-14f786410301-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227134    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a994b38f-0096-4fff-b49c-e6d99dc5a6be-kubelet-config\") pod \"global-pull-secret-syncer-x48xl\" (UID: \"a994b38f-0096-4fff-b49c-e6d99dc5a6be\") " pod="kube-system/global-pull-secret-syncer-x48xl"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227146    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-cnibin\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227092    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-run-netns\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227093    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-system-cni-dir\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227170    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227196    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-cni-multus\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227201    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-cnibin\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227221    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-sys-fs\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227247    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/30efbdba-8299-418b-b8f1-233c789443f5-host-var-lib-cni-multus\") pod \"multus-n5mdk\" (UID: \"30efbdba-8299-418b-b8f1-233c789443f5\") " pod="openshift-multus/multus-n5mdk"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.227256    4865 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227256    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227292    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0aa5b60a-ce95-4623-8ae2-32896ca1232d-sys-fs\") pod \"aws-ebs-csi-driver-node-lrw64\" (UID: \"0aa5b60a-ce95-4623-8ae2-32896ca1232d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lrw64"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227298    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-var-lib-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.227311    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs podName:cc72c8e1-7e7c-4f53-946b-6ef94d6c5971 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.727298063 +0000 UTC m=+1.954084719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs") pod "router-default-56bdbb678c-mvkmq" (UID: "cc72c8e1-7e7c-4f53-946b-6ef94d6c5971") : secret "router-metrics-certs-default" not found
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227337    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-var-lib-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227341    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-cnibin\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.227356    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle podName:cc72c8e1-7e7c-4f53-946b-6ef94d6c5971 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.727347138 +0000 UTC m=+1.954133823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle") pod "router-default-56bdbb678c-mvkmq" (UID: "cc72c8e1-7e7c-4f53-946b-6ef94d6c5971") : configmap references non-existent config key: service-ca.crt
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227371    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c73834fc-41c8-4eea-a739-52d16535890d-cnibin\") pod \"multus-additional-cni-plugins-78wz6\" (UID: \"c73834fc-41c8-4eea-a739-52d16535890d\") " pod="openshift-multus/multus-additional-cni-plugins-78wz6"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227380    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227404    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227419    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-var-lib-kubelet\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227433    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227453    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.227497    4865 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.227507    4865 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c59b47685-9jmnr: secret "image-registry-tls" not found
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227494    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2a74b29-e9ea-4ab9-b155-795d08684fe6-run-openvswitch\") pod \"ovnkube-node-k76ph\" (UID: \"b2a74b29-e9ea-4ab9-b155-795d08684fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.227530    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls podName:870962af-eac0-484d-9267-db8ab42ef13d nodeName:}" failed. No retries permitted until 2026-11-04 01:55:38.727522545 +0000 UTC m=+1.954309191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls") pod "image-registry-6c59b47685-9jmnr" (UID: "870962af-eac0-484d-9267-db8ab42ef13d") : secret "image-registry-tls" not found
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227494    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2522cd43-48a3-40d4-9f8f-b7a0f778abbd-var-lib-kubelet\") pod \"tuned-h4glm\" (UID: \"2522cd43-48a3-40d4-9f8f-b7a0f778abbd\") " pod="openshift-cluster-node-tuning-operator/tuned-h4glm"
Nov 04 01:55:38.227867 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.227583    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/789e64a1-501d-46d9-a58d-14f786410301-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:38.306787 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.306755    4865 scope.go:122] "RemoveContainer" containerID="5a240684c33f12cf1f818a2ddee27b424185731bb59eb98661d7f29c71edadc0"
Nov 04 01:55:38.731755 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.731721    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:38.731926 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.731777    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:38.731926 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.731813    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:38.731926 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.731844    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:38.731926 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.731861    4865 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:38.731926 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.731871    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-8rvcg\" (UID: \"c63f4e36-582d-43be-8575-3b79b7c4756d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:38.731926 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.731904    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:38.731926 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.731921    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls podName:789e64a1-501d-46d9-a58d-14f786410301 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:39.731903525 +0000 UTC m=+2.958690180 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkhvv" (UID: "789e64a1-501d-46d9-a58d-14f786410301") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:38.732219 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.731955    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle podName:cc72c8e1-7e7c-4f53-946b-6ef94d6c5971 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:39.731943084 +0000 UTC m=+2.958729738 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle") pod "router-default-56bdbb678c-mvkmq" (UID: "cc72c8e1-7e7c-4f53-946b-6ef94d6c5971") : configmap references non-existent config key: service-ca.crt
Nov 04 01:55:38.732219 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.731968    4865 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:55:38.732219 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.731982    4865 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:38.732219 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.732004    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls podName:eddb65a3-0fa8-48b5-8ec9-4c939e189be4 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:39.731993915 +0000 UTC m=+2.958780575 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-sr887" (UID: "eddb65a3-0fa8-48b5-8ec9-4c939e189be4") : secret "samples-operator-tls" not found
Nov 04 01:55:38.732219 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.732005    4865 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c59b47685-9jmnr: secret "image-registry-tls" not found
Nov 04 01:55:38.732219 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.731987    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:38.732219 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.732040    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls podName:870962af-eac0-484d-9267-db8ab42ef13d nodeName:}" failed. No retries permitted until 2026-11-04 01:55:39.732029976 +0000 UTC m=+2.958816652 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls") pod "image-registry-6c59b47685-9jmnr" (UID: "870962af-eac0-484d-9267-db8ab42ef13d") : secret "image-registry-tls" not found
Nov 04 01:55:38.732219 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.732045    4865 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:38.732219 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.732056    4865 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8574f7698d-qmq69: secret "image-registry-tls" not found
Nov 04 01:55:38.732219 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.732058    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:38.732219 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.732085    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls podName:74e217e6-636d-4bab-b37f-c44dc0d8fe81 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:39.73207589 +0000 UTC m=+2.958862537 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls") pod "image-registry-8574f7698d-qmq69" (UID: "74e217e6-636d-4bab-b37f-c44dc0d8fe81") : secret "image-registry-tls" not found
Nov 04 01:55:38.732219 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.732114    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs\") pod \"network-metrics-daemon-74j82\" (UID: \"1536e272-39b6-4702-a35d-0130eef3e290\") " pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:38.732219 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.732146    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:38.732219 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.732149    4865 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 01:55:38.732799 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.732297    4865 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:38.732799 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.732340    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls podName:40363d65-7ece-4f05-8668-439a60582a18 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:39.732326705 +0000 UTC m=+2.959113350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wzmdz" (UID: "40363d65-7ece-4f05-8668-439a60582a18") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:38.734404 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.734378    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b461004a-433c-4411-84df-b9aea9f682ab-metrics-tls\") pod \"dns-default-68kbx\" (UID: \"b461004a-433c-4411-84df-b9aea9f682ab\") " pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:38.734514 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.734415    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-metrics-certs\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:38.734633 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.734615    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1536e272-39b6-4702-a35d-0130eef3e290-metrics-certs\") pod \"network-metrics-daemon-74j82\" (UID: \"1536e272-39b6-4702-a35d-0130eef3e290\") " pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:38.734685 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.734655    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c63f4e36-582d-43be-8575-3b79b7c4756d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-8rvcg\" (UID: \"c63f4e36-582d-43be-8575-3b79b7c4756d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:38.830243 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.830212    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ntp6j_1d895e08-f347-4b7c-b63e-26b06a3f6c10/node-ca/0.log"
Nov 04 01:55:38.909121 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.909090    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"
Nov 04 01:55:38.910168 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.910149    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74j82"
Nov 04 01:55:38.916841 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.916819    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:38.917903 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.917884    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:55:38.946963 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.946923    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/2.log"
Nov 04 01:55:38.947493 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.947472    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/1.log"
Nov 04 01:55:38.947579 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.947510    4865 generic.go:356] "Generic (PLEG): container finished" podID="417744b4-5ed7-4b20-9960-7147d2d567ee" containerID="29219ba5858e04d7e00a4eda5de1863bd6e261aa9ca9524f3d674563476b22a7" exitCode=255
Nov 04 01:55:38.948206 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.948177    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg" event={"ID":"417744b4-5ed7-4b20-9960-7147d2d567ee","Type":"ContainerDied","Data":"29219ba5858e04d7e00a4eda5de1863bd6e261aa9ca9524f3d674563476b22a7"}
Nov 04 01:55:38.948323 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.948237    4865 scope.go:122] "RemoveContainer" containerID="5a240684c33f12cf1f818a2ddee27b424185731bb59eb98661d7f29c71edadc0"
Nov 04 01:55:38.962301 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.954582    4865 scope.go:122] "RemoveContainer" containerID="29219ba5858e04d7e00a4eda5de1863bd6e261aa9ca9524f3d674563476b22a7"
Nov 04 01:55:38.962301 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:38.954772    4865 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-jlqmg_openshift-console-operator(417744b4-5ed7-4b20-9960-7147d2d567ee)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg" podUID="417744b4-5ed7-4b20-9960-7147d2d567ee"
Nov 04 01:55:38.962301 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:38.954834    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:39.231548 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:39.231522    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5ljw9_f3db2db6-6f0f-4af7-9bb7-493df9107bb9/serve-healthcheck-canary/0.log"
Nov 04 01:55:39.288717 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:39.288641    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-8rvcg"]
Nov 04 01:55:39.291244 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:39.291208    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc63f4e36_582d_43be_8575_3b79b7c4756d.slice/crio-f50d3318ee32748753b04e84b95100223131926947c85a665fb490973f26068b WatchSource:0}: Error finding container f50d3318ee32748753b04e84b95100223131926947c85a665fb490973f26068b: Status 404 returned error can't find the container with id f50d3318ee32748753b04e84b95100223131926947c85a665fb490973f26068b
Nov 04 01:55:39.293244 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:39.293227    4865 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:55:39.337835 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:39.337598    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-74j82"]
Nov 04 01:55:39.340074 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:39.340050    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1536e272_39b6_4702_a35d_0130eef3e290.slice/crio-a1b25fbb35702ca3c9ca14808226458e215c5f6c863e865ea5a0e7633c7f5629 WatchSource:0}: Error finding container a1b25fbb35702ca3c9ca14808226458e215c5f6c863e865ea5a0e7633c7f5629: Status 404 returned error can't find the container with id a1b25fbb35702ca3c9ca14808226458e215c5f6c863e865ea5a0e7633c7f5629
Nov 04 01:55:39.349976 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:39.349955    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-68kbx"]
Nov 04 01:55:39.352486 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:39.352465    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb461004a_433c_4411_84df_b9aea9f682ab.slice/crio-23c7a68667ae45c7f8900c31c882c4bd80e7e4f04d6b34369e876a08ac447761 WatchSource:0}: Error finding container 23c7a68667ae45c7f8900c31c882c4bd80e7e4f04d6b34369e876a08ac447761: Status 404 returned error can't find the container with id 23c7a68667ae45c7f8900c31c882c4bd80e7e4f04d6b34369e876a08ac447761
Nov 04 01:55:39.739754 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:39.739717    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:39.739754 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:39.739756    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:39.740011 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:39.739782    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:39.740011 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:39.739872    4865 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:39.740011 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:39.739875    4865 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:55:39.740011 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:39.739915    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls podName:40363d65-7ece-4f05-8668-439a60582a18 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:41.73990412 +0000 UTC m=+4.966690782 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wzmdz" (UID: "40363d65-7ece-4f05-8668-439a60582a18") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:39.740011 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:39.739917    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:39.740011 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:39.739877    4865 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:39.740011 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:39.739944    4865 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8574f7698d-qmq69: secret "image-registry-tls" not found
Nov 04 01:55:39.740011 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:39.739946    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls podName:eddb65a3-0fa8-48b5-8ec9-4c939e189be4 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:41.739929341 +0000 UTC m=+4.966715987 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-sr887" (UID: "eddb65a3-0fa8-48b5-8ec9-4c939e189be4") : secret "samples-operator-tls" not found
Nov 04 01:55:39.740011 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:39.739986    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls podName:74e217e6-636d-4bab-b37f-c44dc0d8fe81 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:41.739972645 +0000 UTC m=+4.966759306 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls") pod "image-registry-8574f7698d-qmq69" (UID: "74e217e6-636d-4bab-b37f-c44dc0d8fe81") : secret "image-registry-tls" not found
Nov 04 01:55:39.740378 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:39.740026    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:39.740378 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:39.740060    4865 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:39.740378 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:39.740116    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls podName:789e64a1-501d-46d9-a58d-14f786410301 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:41.740102989 +0000 UTC m=+4.966889649 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkhvv" (UID: "789e64a1-501d-46d9-a58d-14f786410301") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:39.740378 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:39.740058    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:39.740378 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:39.740117    4865 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:39.740378 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:39.740159    4865 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c59b47685-9jmnr: secret "image-registry-tls" not found
Nov 04 01:55:39.740378 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:39.740184    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle podName:cc72c8e1-7e7c-4f53-946b-6ef94d6c5971 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:41.740169529 +0000 UTC m=+4.966956178 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle") pod "router-default-56bdbb678c-mvkmq" (UID: "cc72c8e1-7e7c-4f53-946b-6ef94d6c5971") : configmap references non-existent config key: service-ca.crt
Nov 04 01:55:39.740378 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:39.740204    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls podName:870962af-eac0-484d-9267-db8ab42ef13d nodeName:}" failed. No retries permitted until 2026-11-04 01:55:41.740194818 +0000 UTC m=+4.966981471 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls") pod "image-registry-6c59b47685-9jmnr" (UID: "870962af-eac0-484d-9267-db8ab42ef13d") : secret "image-registry-tls" not found
Nov 04 01:55:39.952372 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:39.952334    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-68kbx" event={"ID":"b461004a-433c-4411-84df-b9aea9f682ab","Type":"ContainerStarted","Data":"23c7a68667ae45c7f8900c31c882c4bd80e7e4f04d6b34369e876a08ac447761"}
Nov 04 01:55:39.953475 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:39.953451    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-74j82" event={"ID":"1536e272-39b6-4702-a35d-0130eef3e290","Type":"ContainerStarted","Data":"a1b25fbb35702ca3c9ca14808226458e215c5f6c863e865ea5a0e7633c7f5629"}
Nov 04 01:55:39.954594 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:39.954574    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg" event={"ID":"c63f4e36-582d-43be-8575-3b79b7c4756d","Type":"ContainerStarted","Data":"f50d3318ee32748753b04e84b95100223131926947c85a665fb490973f26068b"}
Nov 04 01:55:39.955943 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:39.955928    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/2.log"
Nov 04 01:55:39.956232 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:39.956219    4865 scope.go:122] "RemoveContainer" containerID="29219ba5858e04d7e00a4eda5de1863bd6e261aa9ca9524f3d674563476b22a7"
Nov 04 01:55:39.956414 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:39.956398    4865 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-jlqmg_openshift-console-operator(417744b4-5ed7-4b20-9960-7147d2d567ee)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg" podUID="417744b4-5ed7-4b20-9960-7147d2d567ee"
Nov 04 01:55:40.232086 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:40.232050    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-84j4p_d09416ce-b095-4a28-9cb9-c0ad486ec03c/kube-storage-version-migrator-operator/0.log"
Nov 04 01:55:40.962058 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:40.962019    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-8rvcg" event={"ID":"c63f4e36-582d-43be-8575-3b79b7c4756d","Type":"ContainerStarted","Data":"6fbcdbcc2ef2d7931567329d058dd94a05bc9d1c0e23e13bba07b03a78657fd8"}
Nov 04 01:55:41.762546 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:41.762487    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:41.762717 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:41.762565    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:41.762717 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:41.762598    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:41.762717 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:41.762617    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:41.762717 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:41.762664    4865 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:41.762717 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:41.762699    4865 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:41.762717 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:41.762709    4865 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c59b47685-9jmnr: secret "image-registry-tls" not found
Nov 04 01:55:41.762717 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:41.762712    4865 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:41.763051 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:41.762737    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls podName:40363d65-7ece-4f05-8668-439a60582a18 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:45.762717447 +0000 UTC m=+8.989504092 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wzmdz" (UID: "40363d65-7ece-4f05-8668-439a60582a18") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:41.763051 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:41.762755    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls podName:870962af-eac0-484d-9267-db8ab42ef13d nodeName:}" failed. No retries permitted until 2026-11-04 01:55:45.762748019 +0000 UTC m=+8.989534665 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls") pod "image-registry-6c59b47685-9jmnr" (UID: "870962af-eac0-484d-9267-db8ab42ef13d") : secret "image-registry-tls" not found
Nov 04 01:55:41.763051 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:41.762772    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:41.763051 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:41.762806    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:41.763051 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:41.762833    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle podName:cc72c8e1-7e7c-4f53-946b-6ef94d6c5971 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:45.762806371 +0000 UTC m=+8.989593032 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle") pod "router-default-56bdbb678c-mvkmq" (UID: "cc72c8e1-7e7c-4f53-946b-6ef94d6c5971") : configmap references non-existent config key: service-ca.crt
Nov 04 01:55:41.763051 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:41.762869    4865 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:55:41.763051 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:41.762872    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls podName:789e64a1-501d-46d9-a58d-14f786410301 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:45.76286058 +0000 UTC m=+8.989647241 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkhvv" (UID: "789e64a1-501d-46d9-a58d-14f786410301") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:41.763051 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:41.762879    4865 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8574f7698d-qmq69: secret "image-registry-tls" not found
Nov 04 01:55:41.763051 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:41.762910    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls podName:74e217e6-636d-4bab-b37f-c44dc0d8fe81 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:45.76290266 +0000 UTC m=+8.989689305 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls") pod "image-registry-8574f7698d-qmq69" (UID: "74e217e6-636d-4bab-b37f-c44dc0d8fe81") : secret "image-registry-tls" not found
Nov 04 01:55:41.763051 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:41.762874    4865 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:55:41.763051 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:41.762940    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls podName:eddb65a3-0fa8-48b5-8ec9-4c939e189be4 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:45.762934758 +0000 UTC m=+8.989721404 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-sr887" (UID: "eddb65a3-0fa8-48b5-8ec9-4c939e189be4") : secret "samples-operator-tls" not found
Nov 04 01:55:41.967534 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:41.967407    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-74j82" event={"ID":"1536e272-39b6-4702-a35d-0130eef3e290","Type":"ContainerStarted","Data":"60b56982c3b04abe98ee5f46956b60af401d6018b103ad35355dc7d7fb0c28e2"}
Nov 04 01:55:41.967534 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:41.967502    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-74j82" event={"ID":"1536e272-39b6-4702-a35d-0130eef3e290","Type":"ContainerStarted","Data":"af8cead254ac60271c5ef31dffebcd49338177bfe2c4361b48a8edde91189686"}
Nov 04 01:55:42.659984 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:42.659957    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:42.972949 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:42.972908    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-68kbx" event={"ID":"b461004a-433c-4411-84df-b9aea9f682ab","Type":"ContainerStarted","Data":"7b4881031d89424da4ba1fd1852678ea8a783018ceb1a880f1e3124d705518f2"}
Nov 04 01:55:42.973377 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:42.973001    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:42.973377 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:42.973044    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-68kbx" event={"ID":"b461004a-433c-4411-84df-b9aea9f682ab","Type":"ContainerStarted","Data":"f85c5b32c4cc8a17636e1a3fad70fcb1224a2f029e0885e414ddabdcd8641efe"}
Nov 04 01:55:44.731815 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:44.731779    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dszxl"
Nov 04 01:55:45.800457 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:45.800416    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:45.800457 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:45.800462    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:45.801060 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:45.800500    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:45.801060 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:45.800546    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:45.801060 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:45.800556    4865 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:55:45.801060 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:45.800595    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:45.801060 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:45.800617    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls podName:eddb65a3-0fa8-48b5-8ec9-4c939e189be4 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:53.800602209 +0000 UTC m=+17.027388870 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-sr887" (UID: "eddb65a3-0fa8-48b5-8ec9-4c939e189be4") : secret "samples-operator-tls" not found
Nov 04 01:55:45.801060 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:45.800643    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:45.801060 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:45.800656    4865 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:45.801060 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:45.800686    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle podName:cc72c8e1-7e7c-4f53-946b-6ef94d6c5971 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:53.800671244 +0000 UTC m=+17.027457894 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle") pod "router-default-56bdbb678c-mvkmq" (UID: "cc72c8e1-7e7c-4f53-946b-6ef94d6c5971") : configmap references non-existent config key: service-ca.crt
Nov 04 01:55:45.801060 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:45.800710    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls podName:40363d65-7ece-4f05-8668-439a60582a18 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:53.800698605 +0000 UTC m=+17.027485271 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wzmdz" (UID: "40363d65-7ece-4f05-8668-439a60582a18") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:55:45.802993 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:45.802971    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls\") pod \"image-registry-6c59b47685-9jmnr\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") " pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:45.803080 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:45.802975    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/789e64a1-501d-46d9-a58d-14f786410301-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkhvv\" (UID: \"789e64a1-501d-46d9-a58d-14f786410301\") " pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:45.803080 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:45.803067    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls\") pod \"image-registry-8574f7698d-qmq69\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") " pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:45.809035 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:45.809015    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:45.815815 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:45.815795    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:45.818300 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:45.818264    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fkhvv"
Nov 04 01:55:46.281312 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:46.281183    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fkhvv"]
Nov 04 01:55:46.284117 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:46.284087    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod789e64a1_501d_46d9_a58d_14f786410301.slice/crio-ce582872585e1e0261b0ac3992cae51a3d8b687d383d48d9ad441c58e2e35ae7 WatchSource:0}: Error finding container ce582872585e1e0261b0ac3992cae51a3d8b687d383d48d9ad441c58e2e35ae7: Status 404 returned error can't find the container with id ce582872585e1e0261b0ac3992cae51a3d8b687d383d48d9ad441c58e2e35ae7
Nov 04 01:55:46.286406 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:46.286379    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod870962af_eac0_484d_9267_db8ab42ef13d.slice/crio-e094fe85db77a329f5f8eb72c003555c6fc8b76bb2bb75567ad759c16c4092bc WatchSource:0}: Error finding container e094fe85db77a329f5f8eb72c003555c6fc8b76bb2bb75567ad759c16c4092bc: Status 404 returned error can't find the container with id e094fe85db77a329f5f8eb72c003555c6fc8b76bb2bb75567ad759c16c4092bc
Nov 04 01:55:46.332088 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:46.332064    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c59b47685-9jmnr"]
Nov 04 01:55:46.338060 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:46.338033    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8574f7698d-qmq69"]
Nov 04 01:55:46.338355 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:46.338328    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74e217e6_636d_4bab_b37f_c44dc0d8fe81.slice/crio-112bcecd04a50ab654eced9ed5960107dfc7ab5bfec959bf6f6910411330d8a1 WatchSource:0}: Error finding container 112bcecd04a50ab654eced9ed5960107dfc7ab5bfec959bf6f6910411330d8a1: Status 404 returned error can't find the container with id 112bcecd04a50ab654eced9ed5960107dfc7ab5bfec959bf6f6910411330d8a1
Nov 04 01:55:46.974572 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:46.974529    4865 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:46.974958 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:46.974944    4865 scope.go:122] "RemoveContainer" containerID="29219ba5858e04d7e00a4eda5de1863bd6e261aa9ca9524f3d674563476b22a7"
Nov 04 01:55:46.975146 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:46.975125    4865 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-jlqmg_openshift-console-operator(417744b4-5ed7-4b20-9960-7147d2d567ee)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg" podUID="417744b4-5ed7-4b20-9960-7147d2d567ee"
Nov 04 01:55:46.986052 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:46.986016    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8574f7698d-qmq69" event={"ID":"74e217e6-636d-4bab-b37f-c44dc0d8fe81","Type":"ContainerStarted","Data":"604317fbec7cf07c2f269f9d418920e8a53f4bdc2441d2ec1f021fab1d9d9139"}
Nov 04 01:55:46.986341 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:46.986068    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8574f7698d-qmq69" event={"ID":"74e217e6-636d-4bab-b37f-c44dc0d8fe81","Type":"ContainerStarted","Data":"112bcecd04a50ab654eced9ed5960107dfc7ab5bfec959bf6f6910411330d8a1"}
Nov 04 01:55:46.987620 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:46.987588    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c59b47685-9jmnr" event={"ID":"870962af-eac0-484d-9267-db8ab42ef13d","Type":"ContainerStarted","Data":"933e5f610aaecc535944e7194551257b3f623fd1a1eefecd1fc9f40be570dbde"}
Nov 04 01:55:46.987734 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:46.987638    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c59b47685-9jmnr" event={"ID":"870962af-eac0-484d-9267-db8ab42ef13d","Type":"ContainerStarted","Data":"e094fe85db77a329f5f8eb72c003555c6fc8b76bb2bb75567ad759c16c4092bc"}
Nov 04 01:55:46.988937 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:46.988915    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fkhvv" event={"ID":"789e64a1-501d-46d9-a58d-14f786410301","Type":"ContainerStarted","Data":"a805de984c99bf16033c9c4f031112ec23f5aaf7e300bfae77048d4774cc9d97"}
Nov 04 01:55:46.989017 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:46.988962    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fkhvv" event={"ID":"789e64a1-501d-46d9-a58d-14f786410301","Type":"ContainerStarted","Data":"ce582872585e1e0261b0ac3992cae51a3d8b687d383d48d9ad441c58e2e35ae7"}
Nov 04 01:55:47.238517 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:47.238431    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:47.238677 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:47.238649    4865 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:55:47.994979 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:47.994954    4865 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:55:47.995349 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:47.994990    4865 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:55:47.995349 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:47.994952    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fkhvv" event={"ID":"789e64a1-501d-46d9-a58d-14f786410301","Type":"ContainerStarted","Data":"c78abf0892aa11a9f90e379d5a812283b2901eb6d206f74f493587e53d2eb48e"}
Nov 04 01:55:48.854458 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:48.854425    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k76ph"
Nov 04 01:55:48.889093 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:48.889059    4865 scope.go:122] "RemoveContainer" containerID="29219ba5858e04d7e00a4eda5de1863bd6e261aa9ca9524f3d674563476b22a7"
Nov 04 01:55:49.002889 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:49.002863    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/2.log"
Nov 04 01:55:49.003241 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:49.002956    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg" event={"ID":"417744b4-5ed7-4b20-9960-7147d2d567ee","Type":"ContainerStarted","Data":"12e889c9856e067bf52dc04ed314f8d1806ba8fb77ada4c46742d1638cd17bd6"}
Nov 04 01:55:49.003241 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:49.003192    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:49.004240 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:49.004219    4865 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-jlqmg container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.10:8443/readyz\": dial tcp 10.132.0.10:8443: connect: connection refused" start-of-body=
Nov 04 01:55:49.004345 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:49.004259    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg" podUID="417744b4-5ed7-4b20-9960-7147d2d567ee" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.10:8443/readyz\": dial tcp 10.132.0.10:8443: connect: connection refused"
Nov 04 01:55:50.011220 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:50.011188    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-jlqmg"
Nov 04 01:55:51.012387 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:51.012345    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fkhvv" event={"ID":"789e64a1-501d-46d9-a58d-14f786410301","Type":"ContainerStarted","Data":"a39e50cef631f7cd7394ad669ecb726683e2d04e0acef12d1f7a0a8057de6828"}
Nov 04 01:55:52.273756 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:52.273715    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:55:52.333694 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:52.333651    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c59b47685-9jmnr"]
Nov 04 01:55:52.334144 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:52.334123    4865 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:55:52.977795 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:52.977766    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-68kbx"
Nov 04 01:55:53.228709 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:53.228611    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:55:53.870863 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:53.870823    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:53.871368 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:53.870871    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:53.871368 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:53.871004    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:53.871668 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:53.871642    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc72c8e1-7e7c-4f53-946b-6ef94d6c5971-service-ca-bundle\") pod \"router-default-56bdbb678c-mvkmq\" (UID: \"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971\") " pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:53.873241 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:53.873220    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40363d65-7ece-4f05-8668-439a60582a18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wzmdz\" (UID: \"40363d65-7ece-4f05-8668-439a60582a18\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:53.873386 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:53.873301    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eddb65a3-0fa8-48b5-8ec9-4c939e189be4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-sr887\" (UID: \"eddb65a3-0fa8-48b5-8ec9-4c939e189be4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:53.906326 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:53.906295    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"
Nov 04 01:55:53.907386 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:53.907364    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"
Nov 04 01:55:53.908412 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:53.908389    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:54.389209 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:54.389179    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887"]
Nov 04 01:55:54.392539 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:54.392510    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40363d65_7ece_4f05_8668_439a60582a18.slice/crio-a36ed6f9c567c5e7b50f2392c209e8a1f0e6e6fec6dee80c3b1365ac02d6f02b WatchSource:0}: Error finding container a36ed6f9c567c5e7b50f2392c209e8a1f0e6e6fec6dee80c3b1365ac02d6f02b: Status 404 returned error can't find the container with id a36ed6f9c567c5e7b50f2392c209e8a1f0e6e6fec6dee80c3b1365ac02d6f02b
Nov 04 01:55:54.437445 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:55:54.437418    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc72c8e1_7e7c_4f53_946b_6ef94d6c5971.slice/crio-7f44a48d6c7b80f46732f908eb47bf200839fc6357f265ba02ed7f6a97b930eb WatchSource:0}: Error finding container 7f44a48d6c7b80f46732f908eb47bf200839fc6357f265ba02ed7f6a97b930eb: Status 404 returned error can't find the container with id 7f44a48d6c7b80f46732f908eb47bf200839fc6357f265ba02ed7f6a97b930eb
Nov 04 01:55:54.439217 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:54.439185    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz"]
Nov 04 01:55:54.439852 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:54.439830    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-56bdbb678c-mvkmq"]
Nov 04 01:55:55.028118 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:55.028080    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-56bdbb678c-mvkmq" event={"ID":"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971","Type":"ContainerStarted","Data":"6d79f89d7b698462112f966625b176d05bfecbd318cae6012787d1845678f044"}
Nov 04 01:55:55.028599 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:55.028128    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-56bdbb678c-mvkmq" event={"ID":"cc72c8e1-7e7c-4f53-946b-6ef94d6c5971","Type":"ContainerStarted","Data":"7f44a48d6c7b80f46732f908eb47bf200839fc6357f265ba02ed7f6a97b930eb"}
Nov 04 01:55:55.029305 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:55.029266    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz" event={"ID":"40363d65-7ece-4f05-8668-439a60582a18","Type":"ContainerStarted","Data":"a36ed6f9c567c5e7b50f2392c209e8a1f0e6e6fec6dee80c3b1365ac02d6f02b"}
Nov 04 01:55:55.030275 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:55.030253    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887" event={"ID":"eddb65a3-0fa8-48b5-8ec9-4c939e189be4","Type":"ContainerStarted","Data":"14a8b9c6e5fd400a6254e9d3c16605edc625a4dad74f91263ecff1ddbb9c3c3f"}
Nov 04 01:55:55.909547 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:55.909516    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:55.912699 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:55.912670    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:56.033779 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:56.033737    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:56.034987 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:56.034963    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-56bdbb678c-mvkmq"
Nov 04 01:55:57.038559 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:57.038504    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wzmdz" event={"ID":"40363d65-7ece-4f05-8668-439a60582a18","Type":"ContainerStarted","Data":"6fe85ef56bf4c810f89362b6f061ace70f1248aa74588992fbde9a8c8a41daa8"}
Nov 04 01:55:57.042477 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:57.042440    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887" event={"ID":"eddb65a3-0fa8-48b5-8ec9-4c939e189be4","Type":"ContainerStarted","Data":"be75a1adb2d08cc9df515ff1f2679940d718c2237b03b0da320d74fb58cd76e8"}
Nov 04 01:55:57.042612 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:55:57.042513    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-sr887" event={"ID":"eddb65a3-0fa8-48b5-8ec9-4c939e189be4","Type":"ContainerStarted","Data":"f822f97cf8278188e872cf389d8563b487f70f3ef54c9fe51bee26612050a120"}
Nov 04 01:55:58.403711 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:55:58.403670    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 01:56:02.278300 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:02.278258    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:56:03.233349 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:03.233318    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:56:08.950730 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:08.950696    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-g6xmn"
Nov 04 01:56:10.352437 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.352401    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h7xrq"]
Nov 04 01:56:10.355628 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.355610    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.395405 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.395367    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67c30d6f-2598-4efe-9613-37b12c8f0be2-metrics-client-ca\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.395576 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.395405    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67c30d6f-2598-4efe-9613-37b12c8f0be2-node-exporter-wtmp\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.395576 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.395445    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67c30d6f-2598-4efe-9613-37b12c8f0be2-root\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.395576 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.395553    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67c30d6f-2598-4efe-9613-37b12c8f0be2-node-exporter-tls\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.395702 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.395586    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67c30d6f-2598-4efe-9613-37b12c8f0be2-node-exporter-textfile\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.395702 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.395607    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67c30d6f-2598-4efe-9613-37b12c8f0be2-sys\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.395702 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.395684    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jblgr\" (UniqueName: \"kubernetes.io/projected/67c30d6f-2598-4efe-9613-37b12c8f0be2-kube-api-access-jblgr\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.395805 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.395705    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67c30d6f-2598-4efe-9613-37b12c8f0be2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.395805 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.395723    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67c30d6f-2598-4efe-9613-37b12c8f0be2-node-exporter-accelerators-collector-config\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.471856 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.471823    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 04 01:56:10.471856 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.471844    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 04 01:56:10.497002 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.496970    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67c30d6f-2598-4efe-9613-37b12c8f0be2-metrics-client-ca\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.497186 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.497014    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67c30d6f-2598-4efe-9613-37b12c8f0be2-node-exporter-wtmp\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.497186 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.497043    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67c30d6f-2598-4efe-9613-37b12c8f0be2-root\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.497186 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.497135    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67c30d6f-2598-4efe-9613-37b12c8f0be2-node-exporter-tls\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.497186 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.497177    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67c30d6f-2598-4efe-9613-37b12c8f0be2-node-exporter-textfile\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.497419 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.497193    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67c30d6f-2598-4efe-9613-37b12c8f0be2-node-exporter-wtmp\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.497419 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.497211    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67c30d6f-2598-4efe-9613-37b12c8f0be2-sys\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.497419 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.497193    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67c30d6f-2598-4efe-9613-37b12c8f0be2-root\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.497419 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.497258    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67c30d6f-2598-4efe-9613-37b12c8f0be2-sys\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.497419 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.497338    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jblgr\" (UniqueName: \"kubernetes.io/projected/67c30d6f-2598-4efe-9613-37b12c8f0be2-kube-api-access-jblgr\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.497419 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.497363    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67c30d6f-2598-4efe-9613-37b12c8f0be2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.497419 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.497392    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67c30d6f-2598-4efe-9613-37b12c8f0be2-node-exporter-accelerators-collector-config\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.497724 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.497583    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67c30d6f-2598-4efe-9613-37b12c8f0be2-node-exporter-textfile\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.497724 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.497643    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67c30d6f-2598-4efe-9613-37b12c8f0be2-metrics-client-ca\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.497918 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.497901    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67c30d6f-2598-4efe-9613-37b12c8f0be2-node-exporter-accelerators-collector-config\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.522955 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.522923    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 04 01:56:10.523160 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.522937    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 04 01:56:10.523219 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.522989    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hccdr\""
Nov 04 01:56:10.530500 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.530470    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67c30d6f-2598-4efe-9613-37b12c8f0be2-node-exporter-tls\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.530615 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.530543    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67c30d6f-2598-4efe-9613-37b12c8f0be2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.559305 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.559261    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jblgr\" (UniqueName: \"kubernetes.io/projected/67c30d6f-2598-4efe-9613-37b12c8f0be2-kube-api-access-jblgr\") pod \"node-exporter-h7xrq\" (UID: \"67c30d6f-2598-4efe-9613-37b12c8f0be2\") " pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.670034 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:10.669946    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h7xrq"
Nov 04 01:56:10.680384 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:56:10.680357    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67c30d6f_2598_4efe_9613_37b12c8f0be2.slice/crio-3ffc737896896444e7718ec212e8c94c8c5a28ff3d137a9699310324c4a65091 WatchSource:0}: Error finding container 3ffc737896896444e7718ec212e8c94c8c5a28ff3d137a9699310324c4a65091: Status 404 returned error can't find the container with id 3ffc737896896444e7718ec212e8c94c8c5a28ff3d137a9699310324c4a65091
Nov 04 01:56:11.092853 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:11.092789    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h7xrq" event={"ID":"67c30d6f-2598-4efe-9613-37b12c8f0be2","Type":"ContainerStarted","Data":"3ffc737896896444e7718ec212e8c94c8c5a28ff3d137a9699310324c4a65091"}
Nov 04 01:56:13.102797 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:13.102766    4865 generic.go:356] "Generic (PLEG): container finished" podID="67c30d6f-2598-4efe-9613-37b12c8f0be2" containerID="737651cb4bcf8b07205abb054d3a4eff0641f10aaa60139a962d1d7837ac48e3" exitCode=0
Nov 04 01:56:13.103224 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:13.102841    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h7xrq" event={"ID":"67c30d6f-2598-4efe-9613-37b12c8f0be2","Type":"ContainerDied","Data":"737651cb4bcf8b07205abb054d3a4eff0641f10aaa60139a962d1d7837ac48e3"}
Nov 04 01:56:14.109274 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:14.109236    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h7xrq" event={"ID":"67c30d6f-2598-4efe-9613-37b12c8f0be2","Type":"ContainerStarted","Data":"18d504e0d4dfbf16b93e582d3bae14d6e0eb59a91c5768f57f506184e3f9e8fa"}
Nov 04 01:56:14.109721 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:14.109314    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h7xrq" event={"ID":"67c30d6f-2598-4efe-9613-37b12c8f0be2","Type":"ContainerStarted","Data":"cda1ecfd7c3d39073fdbb89b0ef52df5f1802ca1667d63f9b056a4ec348dec21"}
Nov 04 01:56:14.235133 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:14.235038    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h7xrq" podStartSLOduration=2.566687028 podStartE2EDuration="4.235025436s" totalImagesPullingTime="1.597987845s" totalInitContainerRuntime="70.350563ms" isStatefulPod=true podCreationTimestamp="2026-11-04 01:56:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:14.233863922 +0000 UTC m=+37.460650591" watchObservedRunningTime="2026-11-04 01:56:14.235025436 +0000 UTC m=+37.461812156"
Nov 04 01:56:15.143776 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:15.143735    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8574f7698d-qmq69"]
Nov 04 01:56:16.020587 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.020554    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"]
Nov 04 01:56:16.024186 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.024169    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.079590 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.079551    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Nov 04 01:56:16.079770 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.079672    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-kn447\""
Nov 04 01:56:16.079770 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.079729    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Nov 04 01:56:16.130547 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.130510    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Nov 04 01:56:16.130547 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.130526    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Nov 04 01:56:16.130981 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.130963    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Nov 04 01:56:16.131715 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.131697    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"]
Nov 04 01:56:16.144549 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.144520    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa8a869d-eb02-4634-ba03-3da321d7d0ba-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.144948 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.144560    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/aa8a869d-eb02-4634-ba03-3da321d7d0ba-federate-client-tls\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.144948 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.144632    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psfbq\" (UniqueName: \"kubernetes.io/projected/aa8a869d-eb02-4634-ba03-3da321d7d0ba-kube-api-access-psfbq\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.144948 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.144703    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/aa8a869d-eb02-4634-ba03-3da321d7d0ba-telemeter-client-tls\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.144948 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.144764    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa8a869d-eb02-4634-ba03-3da321d7d0ba-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.144948 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.144809    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa8a869d-eb02-4634-ba03-3da321d7d0ba-serving-certs-ca-bundle\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.144948 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.144838    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/aa8a869d-eb02-4634-ba03-3da321d7d0ba-secret-telemeter-client\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.144948 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.144858    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa8a869d-eb02-4634-ba03-3da321d7d0ba-metrics-client-ca\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.233066 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.233031    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Nov 04 01:56:16.245514 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.245481    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa8a869d-eb02-4634-ba03-3da321d7d0ba-serving-certs-ca-bundle\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.245514 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.245516    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/aa8a869d-eb02-4634-ba03-3da321d7d0ba-secret-telemeter-client\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.245735 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.245537    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa8a869d-eb02-4634-ba03-3da321d7d0ba-metrics-client-ca\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.245735 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.245586    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa8a869d-eb02-4634-ba03-3da321d7d0ba-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.245735 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.245603    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/aa8a869d-eb02-4634-ba03-3da321d7d0ba-federate-client-tls\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.245735 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.245628    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psfbq\" (UniqueName: \"kubernetes.io/projected/aa8a869d-eb02-4634-ba03-3da321d7d0ba-kube-api-access-psfbq\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.245735 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.245665    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/aa8a869d-eb02-4634-ba03-3da321d7d0ba-telemeter-client-tls\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.245961 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.245732    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa8a869d-eb02-4634-ba03-3da321d7d0ba-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.246456 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.246428    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa8a869d-eb02-4634-ba03-3da321d7d0ba-serving-certs-ca-bundle\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.246690 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.246667    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa8a869d-eb02-4634-ba03-3da321d7d0ba-metrics-client-ca\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.246813 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.246793    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa8a869d-eb02-4634-ba03-3da321d7d0ba-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.248671 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.248651    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/aa8a869d-eb02-4634-ba03-3da321d7d0ba-federate-client-tls\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.248791 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.248767    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa8a869d-eb02-4634-ba03-3da321d7d0ba-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.248886 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.248872    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/aa8a869d-eb02-4634-ba03-3da321d7d0ba-secret-telemeter-client\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.248929 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.248876    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/aa8a869d-eb02-4634-ba03-3da321d7d0ba-telemeter-client-tls\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.308444 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.308375    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psfbq\" (UniqueName: \"kubernetes.io/projected/aa8a869d-eb02-4634-ba03-3da321d7d0ba-kube-api-access-psfbq\") pod \"telemeter-client-7b55bdf9f-dfr2j\" (UID: \"aa8a869d-eb02-4634-ba03-3da321d7d0ba\") " pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.336504 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.336469    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"
Nov 04 01:56:16.770322 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:16.770299    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j"]
Nov 04 01:56:16.772918 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:56:16.772889    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa8a869d_eb02_4634_ba03_3da321d7d0ba.slice/crio-fe801886edb1694737c26f29d9cf2dd6fc9196fa60214b13aa0a63c53ac04bd3 WatchSource:0}: Error finding container fe801886edb1694737c26f29d9cf2dd6fc9196fa60214b13aa0a63c53ac04bd3: Status 404 returned error can't find the container with id fe801886edb1694737c26f29d9cf2dd6fc9196fa60214b13aa0a63c53ac04bd3
Nov 04 01:56:17.122837 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.122761    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j" event={"ID":"aa8a869d-eb02-4634-ba03-3da321d7d0ba","Type":"ContainerStarted","Data":"fe801886edb1694737c26f29d9cf2dd6fc9196fa60214b13aa0a63c53ac04bd3"}
Nov 04 01:56:17.360428 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.360384    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6c59b47685-9jmnr" podUID="870962af-eac0-484d-9267-db8ab42ef13d" containerName="registry" containerID="cri-o://933e5f610aaecc535944e7194551257b3f623fd1a1eefecd1fc9f40be570dbde" gracePeriod=30
Nov 04 01:56:17.617240 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.617214    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:56:17.658401 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.658322    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-trusted-ca\") pod \"870962af-eac0-484d-9267-db8ab42ef13d\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") "
Nov 04 01:56:17.658401 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.658388    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-image-registry-private-configuration\") pod \"870962af-eac0-484d-9267-db8ab42ef13d\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") "
Nov 04 01:56:17.658640 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.658435    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-bound-sa-token\") pod \"870962af-eac0-484d-9267-db8ab42ef13d\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") "
Nov 04 01:56:17.658640 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.658495    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-registry-certificates\") pod \"870962af-eac0-484d-9267-db8ab42ef13d\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") "
Nov 04 01:56:17.658640 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.658535    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/870962af-eac0-484d-9267-db8ab42ef13d-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/870962af-eac0-484d-9267-db8ab42ef13d-ca-trust-extracted\") pod \"870962af-eac0-484d-9267-db8ab42ef13d\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") "
Nov 04 01:56:17.658800 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.658704    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-installation-pull-secrets\") pod \"870962af-eac0-484d-9267-db8ab42ef13d\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") "
Nov 04 01:56:17.658800 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.658760    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-kube-api-access-9g6rr\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-kube-api-access-9g6rr\") pod \"870962af-eac0-484d-9267-db8ab42ef13d\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") "
Nov 04 01:56:17.658902 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.658813    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls\") pod \"870962af-eac0-484d-9267-db8ab42ef13d\" (UID: \"870962af-eac0-484d-9267-db8ab42ef13d\") "
Nov 04 01:56:17.659030 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.658999    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-registry-certificates" pod "870962af-eac0-484d-9267-db8ab42ef13d" (UID: "870962af-eac0-484d-9267-db8ab42ef13d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:17.659329 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.659302    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-trusted-ca" pod "870962af-eac0-484d-9267-db8ab42ef13d" (UID: "870962af-eac0-484d-9267-db8ab42ef13d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:17.661503 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.661470    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-image-registry-private-configuration" pod "870962af-eac0-484d-9267-db8ab42ef13d" (UID: "870962af-eac0-484d-9267-db8ab42ef13d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:56:17.661608 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.661558    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-bound-sa-token" pod "870962af-eac0-484d-9267-db8ab42ef13d" (UID: "870962af-eac0-484d-9267-db8ab42ef13d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:17.661756 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.661716    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-installation-pull-secrets" pod "870962af-eac0-484d-9267-db8ab42ef13d" (UID: "870962af-eac0-484d-9267-db8ab42ef13d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:56:17.662053 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.662028    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-kube-api-access-9g6rr" pod "870962af-eac0-484d-9267-db8ab42ef13d" (UID: "870962af-eac0-484d-9267-db8ab42ef13d"). InnerVolumeSpecName "kube-api-access-9g6rr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:17.664435 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.664379    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls" pod "870962af-eac0-484d-9267-db8ab42ef13d" (UID: "870962af-eac0-484d-9267-db8ab42ef13d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:17.667097 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.667075    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/870962af-eac0-484d-9267-db8ab42ef13d-ca-trust-extracted" pod "870962af-eac0-484d-9267-db8ab42ef13d" (UID: "870962af-eac0-484d-9267-db8ab42ef13d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:17.760509 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.760453    4865 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-installation-pull-secrets\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:17.760509 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.760509    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9g6rr\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-kube-api-access-9g6rr\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:17.760690 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.760526    4865 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-registry-tls\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:17.760690 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.760541    4865 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-trusted-ca\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:17.760690 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.760557    4865 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/870962af-eac0-484d-9267-db8ab42ef13d-image-registry-private-configuration\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:17.760690 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.760571    4865 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/870962af-eac0-484d-9267-db8ab42ef13d-bound-sa-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:17.760690 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.760585    4865 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/870962af-eac0-484d-9267-db8ab42ef13d-registry-certificates\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:17.760690 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:17.760599    4865 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/870962af-eac0-484d-9267-db8ab42ef13d-ca-trust-extracted\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:18.127821 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:18.127782    4865 generic.go:356] "Generic (PLEG): container finished" podID="870962af-eac0-484d-9267-db8ab42ef13d" containerID="933e5f610aaecc535944e7194551257b3f623fd1a1eefecd1fc9f40be570dbde" exitCode=0
Nov 04 01:56:18.127994 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:18.127831    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c59b47685-9jmnr" event={"ID":"870962af-eac0-484d-9267-db8ab42ef13d","Type":"ContainerDied","Data":"933e5f610aaecc535944e7194551257b3f623fd1a1eefecd1fc9f40be570dbde"}
Nov 04 01:56:18.127994 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:18.127873    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c59b47685-9jmnr" event={"ID":"870962af-eac0-484d-9267-db8ab42ef13d","Type":"ContainerDied","Data":"e094fe85db77a329f5f8eb72c003555c6fc8b76bb2bb75567ad759c16c4092bc"}
Nov 04 01:56:18.127994 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:18.127896    4865 scope.go:122] "RemoveContainer" containerID="933e5f610aaecc535944e7194551257b3f623fd1a1eefecd1fc9f40be570dbde"
Nov 04 01:56:18.127994 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:18.127831    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c59b47685-9jmnr"
Nov 04 01:56:18.140062 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:18.138644    4865 scope.go:122] "RemoveContainer" containerID="933e5f610aaecc535944e7194551257b3f623fd1a1eefecd1fc9f40be570dbde"
Nov 04 01:56:18.140062 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:56:18.139143    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"933e5f610aaecc535944e7194551257b3f623fd1a1eefecd1fc9f40be570dbde\": container with ID starting with 933e5f610aaecc535944e7194551257b3f623fd1a1eefecd1fc9f40be570dbde not found: ID does not exist" containerID="933e5f610aaecc535944e7194551257b3f623fd1a1eefecd1fc9f40be570dbde"
Nov 04 01:56:18.140062 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:18.139181    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"933e5f610aaecc535944e7194551257b3f623fd1a1eefecd1fc9f40be570dbde"} err="failed to get container status \"933e5f610aaecc535944e7194551257b3f623fd1a1eefecd1fc9f40be570dbde\": rpc error: code = NotFound desc = could not find container \"933e5f610aaecc535944e7194551257b3f623fd1a1eefecd1fc9f40be570dbde\": container with ID starting with 933e5f610aaecc535944e7194551257b3f623fd1a1eefecd1fc9f40be570dbde not found: ID does not exist"
Nov 04 01:56:18.296845 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:18.296812    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c59b47685-9jmnr"]
Nov 04 01:56:18.348186 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:18.348154    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6c59b47685-9jmnr"]
Nov 04 01:56:18.894396 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:18.894361    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="870962af-eac0-484d-9267-db8ab42ef13d" path="/var/lib/kubelet/pods/870962af-eac0-484d-9267-db8ab42ef13d/volumes"
Nov 04 01:56:20.138495 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:20.138453    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j" event={"ID":"aa8a869d-eb02-4634-ba03-3da321d7d0ba","Type":"ContainerStarted","Data":"245daef31fb593d1f3f736538aeebdaeecede030f4ff736d21972e0a75551bfd"}
Nov 04 01:56:21.144226 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:21.144189    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j" event={"ID":"aa8a869d-eb02-4634-ba03-3da321d7d0ba","Type":"ContainerStarted","Data":"55e4aba674cb62b22ad6a2965572615f15a50276e7e88eed856784793a43d25a"}
Nov 04 01:56:21.144636 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:21.144246    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j" event={"ID":"aa8a869d-eb02-4634-ba03-3da321d7d0ba","Type":"ContainerStarted","Data":"2193601e9444fbbdff8af04e1ea010c9101dcbc955d2a510059b843d1872dc56"}
Nov 04 01:56:21.371379 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:21.371327    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7b55bdf9f-dfr2j" podStartSLOduration=2.635483232 podStartE2EDuration="6.371314259s" totalImagesPullingTime="3.735831027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:56:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:21.320186384 +0000 UTC m=+44.546973051" watchObservedRunningTime="2026-11-04 01:56:21.371314259 +0000 UTC m=+44.598100923"
Nov 04 01:56:28.402815 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:56:28.402776    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 01:56:40.164059 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.163995    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8574f7698d-qmq69" podUID="74e217e6-636d-4bab-b37f-c44dc0d8fe81" containerName="registry" containerID="cri-o://604317fbec7cf07c2f269f9d418920e8a53f4bdc2441d2ec1f021fab1d9d9139" gracePeriod=30
Nov 04 01:56:40.415665 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.415612    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:56:40.556150 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.556118    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-image-registry-private-configuration\") pod \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") "
Nov 04 01:56:40.556331 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.556187    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-certificates\") pod \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") "
Nov 04 01:56:40.556331 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.556219    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74e217e6-636d-4bab-b37f-c44dc0d8fe81-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/74e217e6-636d-4bab-b37f-c44dc0d8fe81-ca-trust-extracted\") pod \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") "
Nov 04 01:56:40.556331 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.556265    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-kube-api-access-k9hwp\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-kube-api-access-k9hwp\") pod \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") "
Nov 04 01:56:40.556515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.556389    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-trusted-ca\") pod \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") "
Nov 04 01:56:40.556515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.556441    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-installation-pull-secrets\") pod \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") "
Nov 04 01:56:40.556515 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.556472    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-bound-sa-token\") pod \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") "
Nov 04 01:56:40.556668 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.556525    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls\") pod \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\" (UID: \"74e217e6-636d-4bab-b37f-c44dc0d8fe81\") "
Nov 04 01:56:40.556787 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.556653    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-certificates" pod "74e217e6-636d-4bab-b37f-c44dc0d8fe81" (UID: "74e217e6-636d-4bab-b37f-c44dc0d8fe81"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:40.556977 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.556949    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-trusted-ca" pod "74e217e6-636d-4bab-b37f-c44dc0d8fe81" (UID: "74e217e6-636d-4bab-b37f-c44dc0d8fe81"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:40.558714 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.558691    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-installation-pull-secrets" pod "74e217e6-636d-4bab-b37f-c44dc0d8fe81" (UID: "74e217e6-636d-4bab-b37f-c44dc0d8fe81"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:56:40.558869 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.558819    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-image-registry-private-configuration" pod "74e217e6-636d-4bab-b37f-c44dc0d8fe81" (UID: "74e217e6-636d-4bab-b37f-c44dc0d8fe81"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:56:40.558945 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.558932    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls" pod "74e217e6-636d-4bab-b37f-c44dc0d8fe81" (UID: "74e217e6-636d-4bab-b37f-c44dc0d8fe81"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:40.559095 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.559076    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-kube-api-access-k9hwp" pod "74e217e6-636d-4bab-b37f-c44dc0d8fe81" (UID: "74e217e6-636d-4bab-b37f-c44dc0d8fe81"). InnerVolumeSpecName "kube-api-access-k9hwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:40.559399 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.559375    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-bound-sa-token" pod "74e217e6-636d-4bab-b37f-c44dc0d8fe81" (UID: "74e217e6-636d-4bab-b37f-c44dc0d8fe81"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:40.563458 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.563436    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74e217e6-636d-4bab-b37f-c44dc0d8fe81-ca-trust-extracted" pod "74e217e6-636d-4bab-b37f-c44dc0d8fe81" (UID: "74e217e6-636d-4bab-b37f-c44dc0d8fe81"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:40.657404 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.657381    4865 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-trusted-ca\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:40.657404 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.657404    4865 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-installation-pull-secrets\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:40.657566 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.657415    4865 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-bound-sa-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:40.657566 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.657426    4865 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-tls\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:40.657566 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.657436    4865 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/74e217e6-636d-4bab-b37f-c44dc0d8fe81-image-registry-private-configuration\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:40.657566 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.657480    4865 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/74e217e6-636d-4bab-b37f-c44dc0d8fe81-registry-certificates\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:40.657566 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.657489    4865 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/74e217e6-636d-4bab-b37f-c44dc0d8fe81-ca-trust-extracted\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:40.657566 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:40.657497    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9hwp\" (UniqueName: \"kubernetes.io/projected/74e217e6-636d-4bab-b37f-c44dc0d8fe81-kube-api-access-k9hwp\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:41.222167 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:41.222132    4865 generic.go:356] "Generic (PLEG): container finished" podID="74e217e6-636d-4bab-b37f-c44dc0d8fe81" containerID="604317fbec7cf07c2f269f9d418920e8a53f4bdc2441d2ec1f021fab1d9d9139" exitCode=0
Nov 04 01:56:41.222646 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:41.222178    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8574f7698d-qmq69"
Nov 04 01:56:41.222646 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:41.222227    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8574f7698d-qmq69" event={"ID":"74e217e6-636d-4bab-b37f-c44dc0d8fe81","Type":"ContainerDied","Data":"604317fbec7cf07c2f269f9d418920e8a53f4bdc2441d2ec1f021fab1d9d9139"}
Nov 04 01:56:41.222646 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:41.222278    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8574f7698d-qmq69" event={"ID":"74e217e6-636d-4bab-b37f-c44dc0d8fe81","Type":"ContainerDied","Data":"112bcecd04a50ab654eced9ed5960107dfc7ab5bfec959bf6f6910411330d8a1"}
Nov 04 01:56:41.222646 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:41.222313    4865 scope.go:122] "RemoveContainer" containerID="604317fbec7cf07c2f269f9d418920e8a53f4bdc2441d2ec1f021fab1d9d9139"
Nov 04 01:56:41.230215 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:41.230197    4865 scope.go:122] "RemoveContainer" containerID="604317fbec7cf07c2f269f9d418920e8a53f4bdc2441d2ec1f021fab1d9d9139"
Nov 04 01:56:41.230501 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:56:41.230484    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"604317fbec7cf07c2f269f9d418920e8a53f4bdc2441d2ec1f021fab1d9d9139\": container with ID starting with 604317fbec7cf07c2f269f9d418920e8a53f4bdc2441d2ec1f021fab1d9d9139 not found: ID does not exist" containerID="604317fbec7cf07c2f269f9d418920e8a53f4bdc2441d2ec1f021fab1d9d9139"
Nov 04 01:56:41.230566 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:41.230509    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"604317fbec7cf07c2f269f9d418920e8a53f4bdc2441d2ec1f021fab1d9d9139"} err="failed to get container status \"604317fbec7cf07c2f269f9d418920e8a53f4bdc2441d2ec1f021fab1d9d9139\": rpc error: code = NotFound desc = could not find container \"604317fbec7cf07c2f269f9d418920e8a53f4bdc2441d2ec1f021fab1d9d9139\": container with ID starting with 604317fbec7cf07c2f269f9d418920e8a53f4bdc2441d2ec1f021fab1d9d9139 not found: ID does not exist"
Nov 04 01:56:41.441548 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:41.441519    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8574f7698d-qmq69"]
Nov 04 01:56:41.493058 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:41.492979    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8574f7698d-qmq69"]
Nov 04 01:56:42.893650 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:42.893617    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74e217e6-636d-4bab-b37f-c44dc0d8fe81" path="/var/lib/kubelet/pods/74e217e6-636d-4bab-b37f-c44dc0d8fe81/volumes"
Nov 04 01:56:45.239177 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:45.239151    4865 generic.go:356] "Generic (PLEG): container finished" podID="d09416ce-b095-4a28-9cb9-c0ad486ec03c" containerID="a4442abda16fbcd14764c8106809dac533129cd017ab86de08ac4aed3c790f97" exitCode=0
Nov 04 01:56:45.239544 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:45.239185    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p" event={"ID":"d09416ce-b095-4a28-9cb9-c0ad486ec03c","Type":"ContainerDied","Data":"a4442abda16fbcd14764c8106809dac533129cd017ab86de08ac4aed3c790f97"}
Nov 04 01:56:45.239544 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:45.239471    4865 scope.go:122] "RemoveContainer" containerID="a4442abda16fbcd14764c8106809dac533129cd017ab86de08ac4aed3c790f97"
Nov 04 01:56:46.244110 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:46.244074    4865 generic.go:356] "Generic (PLEG): container finished" podID="698e630d-a5b0-43ce-9a12-aac9c60ab53c" containerID="9d66a760b2ca4602a40f00ed106ea89ea3fb426ed38d607fa866e07a23969d55" exitCode=0
Nov 04 01:56:46.244622 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:46.244152    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t" event={"ID":"698e630d-a5b0-43ce-9a12-aac9c60ab53c","Type":"ContainerDied","Data":"9d66a760b2ca4602a40f00ed106ea89ea3fb426ed38d607fa866e07a23969d55"}
Nov 04 01:56:46.244622 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:46.244548    4865 scope.go:122] "RemoveContainer" containerID="9d66a760b2ca4602a40f00ed106ea89ea3fb426ed38d607fa866e07a23969d55"
Nov 04 01:56:46.246024 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:46.246006    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-84j4p" event={"ID":"d09416ce-b095-4a28-9cb9-c0ad486ec03c","Type":"ContainerStarted","Data":"777da79e51b9df3662e870ae25a5a8636f32a966222007e43435e764255f3e6f"}
Nov 04 01:56:47.251981 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:47.251947    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-wzh4t" event={"ID":"698e630d-a5b0-43ce-9a12-aac9c60ab53c","Type":"ContainerStarted","Data":"898034ebe04d12898574dde5826131d28176a2bc2db7a9d914e0374769de2bc7"}
Nov 04 01:56:58.402099 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:56:58.402064    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 01:56:59.296992 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:59.296958    4865 generic.go:356] "Generic (PLEG): container finished" podID="a8ecf9a5-a6e5-460b-8cc9-44c996d57930" containerID="c5729e59dae23db2ab5e4e64cccd538f9b4f94a917a6697f37026a8be5b20e7d" exitCode=0
Nov 04 01:56:59.297161 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:59.297000    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-2bzkj" event={"ID":"a8ecf9a5-a6e5-460b-8cc9-44c996d57930","Type":"ContainerDied","Data":"c5729e59dae23db2ab5e4e64cccd538f9b4f94a917a6697f37026a8be5b20e7d"}
Nov 04 01:56:59.297349 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:56:59.297325    4865 scope.go:122] "RemoveContainer" containerID="c5729e59dae23db2ab5e4e64cccd538f9b4f94a917a6697f37026a8be5b20e7d"
Nov 04 01:57:00.302718 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:57:00.302679    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-2bzkj" event={"ID":"a8ecf9a5-a6e5-460b-8cc9-44c996d57930","Type":"ContainerStarted","Data":"2b4c3e54f855632c80aff3232a5d13de2f901ee2765041345a5ed555da40eba9"}
Nov 04 01:57:28.402307 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:57:28.402249    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 01:57:58.404810 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:57:58.404725    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 01:58:28.403211 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:58:28.403171    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 01:58:58.407001 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:58:58.406956    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 01:59:28.402902 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:59:28.402860    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 01:59:57.703912 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.703871    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk"]
Nov 04 01:59:57.706483 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.704339    4865 cpu_manager.go:414] "containerMap: removing container" podUID="870962af-eac0-484d-9267-db8ab42ef13d" containerName="registry"
Nov 04 01:59:57.706483 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.704379    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="870962af-eac0-484d-9267-db8ab42ef13d" containerName="registry"
Nov 04 01:59:57.706483 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.704408    4865 cpu_manager.go:414] "containerMap: removing container" podUID="74e217e6-636d-4bab-b37f-c44dc0d8fe81" containerName="registry"
Nov 04 01:59:57.706483 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.704413    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74e217e6-636d-4bab-b37f-c44dc0d8fe81" containerName="registry"
Nov 04 01:59:57.706483 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.704480    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="870962af-eac0-484d-9267-db8ab42ef13d" containerName="registry"
Nov 04 01:59:57.706483 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.704492    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="74e217e6-636d-4bab-b37f-c44dc0d8fe81" containerName="registry"
Nov 04 01:59:57.707372 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.707352    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk"
Nov 04 01:59:57.800018 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.799980    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e510dbfb-6713-4b71-a507-c1a5b56db93e-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk\" (UID: \"e510dbfb-6713-4b71-a507-c1a5b56db93e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk"
Nov 04 01:59:57.800158 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.800058    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j9wf\" (UniqueName: \"kubernetes.io/projected/e510dbfb-6713-4b71-a507-c1a5b56db93e-kube-api-access-4j9wf\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk\" (UID: \"e510dbfb-6713-4b71-a507-c1a5b56db93e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk"
Nov 04 01:59:57.800158 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.800129    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e510dbfb-6713-4b71-a507-c1a5b56db93e-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk\" (UID: \"e510dbfb-6713-4b71-a507-c1a5b56db93e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk"
Nov 04 01:59:57.815709 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.815679    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 04 01:59:57.815873 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.815788    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 04 01:59:57.815873 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.815806    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-4sn7l\""
Nov 04 01:59:57.816774 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.816750    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk"]
Nov 04 01:59:57.868702 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.868667    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"]
Nov 04 01:59:57.871707 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.871691    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:57.901131 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.901105    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e510dbfb-6713-4b71-a507-c1a5b56db93e-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk\" (UID: \"e510dbfb-6713-4b71-a507-c1a5b56db93e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk"
Nov 04 01:59:57.901269 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.901146    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bb7c\" (UniqueName: \"kubernetes.io/projected/73c19691-9fba-462f-9b0b-66b093d8f3a2-kube-api-access-9bb7c\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:57.901269 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.901166    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/73c19691-9fba-462f-9b0b-66b093d8f3a2-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:57.901269 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.901183    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:57.901436 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.901319    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e510dbfb-6713-4b71-a507-c1a5b56db93e-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk\" (UID: \"e510dbfb-6713-4b71-a507-c1a5b56db93e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk"
Nov 04 01:59:57.901436 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.901339    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j9wf\" (UniqueName: \"kubernetes.io/projected/e510dbfb-6713-4b71-a507-c1a5b56db93e-kube-api-access-4j9wf\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk\" (UID: \"e510dbfb-6713-4b71-a507-c1a5b56db93e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk"
Nov 04 01:59:57.901436 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.901371    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:57.901436 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.901388    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:57.901436 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.901414    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:57.901674 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.901442    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e510dbfb-6713-4b71-a507-c1a5b56db93e-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk\" (UID: \"e510dbfb-6713-4b71-a507-c1a5b56db93e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk"
Nov 04 01:59:57.901674 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.901456    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:57.901674 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.901646    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e510dbfb-6713-4b71-a507-c1a5b56db93e-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk\" (UID: \"e510dbfb-6713-4b71-a507-c1a5b56db93e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk"
Nov 04 01:59:57.927540 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.927517    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 04 01:59:57.927905 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.927890    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 04 01:59:57.928078 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.928063    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 04 01:59:57.928115 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.928066    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 04 01:59:57.928115 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.928079    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zf425\""
Nov 04 01:59:57.932525 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.932499    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"]
Nov 04 01:59:57.964700 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:57.964651    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j9wf\" (UniqueName: \"kubernetes.io/projected/e510dbfb-6713-4b71-a507-c1a5b56db93e-kube-api-access-4j9wf\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk\" (UID: \"e510dbfb-6713-4b71-a507-c1a5b56db93e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk"
Nov 04 01:59:58.002540 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.002515    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:58.002653 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.002541    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:58.002653 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.002559    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:58.002653 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.002578    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:58.002653 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.002633    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bb7c\" (UniqueName: \"kubernetes.io/projected/73c19691-9fba-462f-9b0b-66b093d8f3a2-kube-api-access-9bb7c\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:58.002848 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.002661    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/73c19691-9fba-462f-9b0b-66b093d8f3a2-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:58.002848 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.002686    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:58.003452 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.003428    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:58.004944 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.004919    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:58.004944 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.004933    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/73c19691-9fba-462f-9b0b-66b093d8f3a2-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:58.005091 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.004921    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:58.005091 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.005002    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:58.022143 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.022125    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk"
Nov 04 01:59:58.064701 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.064665    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:58.064831 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.064809    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bb7c\" (UniqueName: \"kubernetes.io/projected/73c19691-9fba-462f-9b0b-66b093d8f3a2-kube-api-access-9bb7c\") pod \"istiod-openshift-gateway-7876bfcb74-vrvfp\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:58.182613 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.182577    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 01:59:58.391077 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.391045    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk"]
Nov 04 01:59:58.394790 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:59:58.394761    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode510dbfb_6713_4b71_a507_c1a5b56db93e.slice/crio-df7858dcb75032343e1a86adc222d01b882493864524377694137130ebf868c2 WatchSource:0}: Error finding container df7858dcb75032343e1a86adc222d01b882493864524377694137130ebf868c2: Status 404 returned error can't find the container with id df7858dcb75032343e1a86adc222d01b882493864524377694137130ebf868c2
Nov 04 01:59:58.404846 ip-10-0-134-204 kubenswrapper[4865]: E1104 01:59:58.404818    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 01:59:58.558916 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.558894    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"]
Nov 04 01:59:58.561183 ip-10-0-134-204 kubenswrapper[4865]: W1104 01:59:58.561164    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c19691_9fba_462f_9b0b_66b093d8f3a2.slice/crio-f4ca6ec906326cd5ac4e130ccc0efbe929367b33ec67366e91e76e4928a19c18 WatchSource:0}: Error finding container f4ca6ec906326cd5ac4e130ccc0efbe929367b33ec67366e91e76e4928a19c18: Status 404 returned error can't find the container with id f4ca6ec906326cd5ac4e130ccc0efbe929367b33ec67366e91e76e4928a19c18
Nov 04 01:59:58.930464 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.930385    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp" event={"ID":"73c19691-9fba-462f-9b0b-66b093d8f3a2","Type":"ContainerStarted","Data":"f4ca6ec906326cd5ac4e130ccc0efbe929367b33ec67366e91e76e4928a19c18"}
Nov 04 01:59:58.931526 ip-10-0-134-204 kubenswrapper[4865]: I1104 01:59:58.931500    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk" event={"ID":"e510dbfb-6713-4b71-a507-c1a5b56db93e","Type":"ContainerStarted","Data":"df7858dcb75032343e1a86adc222d01b882493864524377694137130ebf868c2"}
Nov 04 02:00:03.262564 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:03.262493    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:00:03.263096 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:03.262598    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:00:03.954129 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:03.954091    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp" event={"ID":"73c19691-9fba-462f-9b0b-66b093d8f3a2","Type":"ContainerStarted","Data":"9b181a0527d810925713e4a3f0d2208ca8ff7207677b29141a6f37a7178078d7"}
Nov 04 02:00:03.954304 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:03.954233    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 02:00:04.188498 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:04.188429    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp" podStartSLOduration=2.489947006 podStartE2EDuration="7.18840791s" totalImagesPullingTime="4.698460904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:59:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:00:04.137066193 +0000 UTC m=+267.363852862" watchObservedRunningTime="2026-11-04 02:00:04.18840791 +0000 UTC m=+267.415194580"
Nov 04 02:00:04.960828 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:04.960794    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 02:00:06.968312 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:06.968267    4865 generic.go:356] "Generic (PLEG): container finished" podID="e510dbfb-6713-4b71-a507-c1a5b56db93e" containerID="1bf207237ef6cbc745f6f15a139ab0fa6cf25daedf62ab75cb18c9c08091e0ae" exitCode=0
Nov 04 02:00:06.968703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:06.968315    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk" event={"ID":"e510dbfb-6713-4b71-a507-c1a5b56db93e","Type":"ContainerDied","Data":"1bf207237ef6cbc745f6f15a139ab0fa6cf25daedf62ab75cb18c9c08091e0ae"}
Nov 04 02:00:08.977572 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:08.977531    4865 generic.go:356] "Generic (PLEG): container finished" podID="e510dbfb-6713-4b71-a507-c1a5b56db93e" containerID="f7363d71dca7af0b9c37d5fc18a3e5352e8ff83ad0fafcf243269fe9a165d4ed" exitCode=0
Nov 04 02:00:08.977988 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:08.977632    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk" event={"ID":"e510dbfb-6713-4b71-a507-c1a5b56db93e","Type":"ContainerDied","Data":"f7363d71dca7af0b9c37d5fc18a3e5352e8ff83ad0fafcf243269fe9a165d4ed"}
Nov 04 02:00:24.042686 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:24.042649    4865 generic.go:356] "Generic (PLEG): container finished" podID="e510dbfb-6713-4b71-a507-c1a5b56db93e" containerID="8dbdd105356c2f7333e52fa9135480260c65b20b19f8cecfba8c59e766f65d1e" exitCode=0
Nov 04 02:00:24.043067 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:24.042700    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk" event={"ID":"e510dbfb-6713-4b71-a507-c1a5b56db93e","Type":"ContainerDied","Data":"8dbdd105356c2f7333e52fa9135480260c65b20b19f8cecfba8c59e766f65d1e"}
Nov 04 02:00:25.166222 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:25.166200    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk"
Nov 04 02:00:25.346768 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:25.346697    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e510dbfb-6713-4b71-a507-c1a5b56db93e-kube-api-access-4j9wf\" (UniqueName: \"kubernetes.io/projected/e510dbfb-6713-4b71-a507-c1a5b56db93e-kube-api-access-4j9wf\") pod \"e510dbfb-6713-4b71-a507-c1a5b56db93e\" (UID: \"e510dbfb-6713-4b71-a507-c1a5b56db93e\") "
Nov 04 02:00:25.346768 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:25.346748    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e510dbfb-6713-4b71-a507-c1a5b56db93e-bundle\" (UniqueName: \"kubernetes.io/empty-dir/e510dbfb-6713-4b71-a507-c1a5b56db93e-bundle\") pod \"e510dbfb-6713-4b71-a507-c1a5b56db93e\" (UID: \"e510dbfb-6713-4b71-a507-c1a5b56db93e\") "
Nov 04 02:00:25.346944 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:25.346772    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e510dbfb-6713-4b71-a507-c1a5b56db93e-util\" (UniqueName: \"kubernetes.io/empty-dir/e510dbfb-6713-4b71-a507-c1a5b56db93e-util\") pod \"e510dbfb-6713-4b71-a507-c1a5b56db93e\" (UID: \"e510dbfb-6713-4b71-a507-c1a5b56db93e\") "
Nov 04 02:00:25.347436 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:25.347406    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e510dbfb-6713-4b71-a507-c1a5b56db93e-bundle" pod "e510dbfb-6713-4b71-a507-c1a5b56db93e" (UID: "e510dbfb-6713-4b71-a507-c1a5b56db93e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:25.348904 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:25.348879    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e510dbfb-6713-4b71-a507-c1a5b56db93e-kube-api-access-4j9wf" pod "e510dbfb-6713-4b71-a507-c1a5b56db93e" (UID: "e510dbfb-6713-4b71-a507-c1a5b56db93e"). InnerVolumeSpecName "kube-api-access-4j9wf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:25.351451 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:25.351425    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e510dbfb-6713-4b71-a507-c1a5b56db93e-util" pod "e510dbfb-6713-4b71-a507-c1a5b56db93e" (UID: "e510dbfb-6713-4b71-a507-c1a5b56db93e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:25.447921 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:25.447886    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4j9wf\" (UniqueName: \"kubernetes.io/projected/e510dbfb-6713-4b71-a507-c1a5b56db93e-kube-api-access-4j9wf\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:25.447921 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:25.447913    4865 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e510dbfb-6713-4b71-a507-c1a5b56db93e-bundle\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:25.447921 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:25.447923    4865 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e510dbfb-6713-4b71-a507-c1a5b56db93e-util\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:26.052276 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:26.052246    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk"
Nov 04 02:00:26.052276 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:26.052263    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rmtlk" event={"ID":"e510dbfb-6713-4b71-a507-c1a5b56db93e","Type":"ContainerDied","Data":"df7858dcb75032343e1a86adc222d01b882493864524377694137130ebf868c2"}
Nov 04 02:00:26.052538 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:26.052320    4865 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df7858dcb75032343e1a86adc222d01b882493864524377694137130ebf868c2"
Nov 04 02:00:28.402700 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:00:28.402651    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:00:36.905233 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:36.905200    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/2.log"
Nov 04 02:00:36.907167 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:36.907148    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/2.log"
Nov 04 02:00:47.073272 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.073183    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"]
Nov 04 02:00:47.073771 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.073740    4865 cpu_manager.go:414] "containerMap: removing container" podUID="e510dbfb-6713-4b71-a507-c1a5b56db93e" containerName="pull"
Nov 04 02:00:47.073771 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.073760    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e510dbfb-6713-4b71-a507-c1a5b56db93e" containerName="pull"
Nov 04 02:00:47.073888 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.073790    4865 cpu_manager.go:414] "containerMap: removing container" podUID="e510dbfb-6713-4b71-a507-c1a5b56db93e" containerName="util"
Nov 04 02:00:47.073888 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.073799    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e510dbfb-6713-4b71-a507-c1a5b56db93e" containerName="util"
Nov 04 02:00:47.073888 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.073815    4865 cpu_manager.go:414] "containerMap: removing container" podUID="e510dbfb-6713-4b71-a507-c1a5b56db93e" containerName="extract"
Nov 04 02:00:47.073888 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.073822    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e510dbfb-6713-4b71-a507-c1a5b56db93e" containerName="extract"
Nov 04 02:00:47.074082 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.073922    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="e510dbfb-6713-4b71-a507-c1a5b56db93e" containerName="extract"
Nov 04 02:00:47.075676 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.075655    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.109547 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.109503    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8cff3a26-844e-4c57-a964-52cb78c0c894-metrics-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.109692 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.109565    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8cff3a26-844e-4c57-a964-52cb78c0c894-manifests\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.109692 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.109667    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8cff3a26-844e-4c57-a964-52cb78c0c894-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.109769 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.109703    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z95q\" (UniqueName: \"kubernetes.io/projected/8cff3a26-844e-4c57-a964-52cb78c0c894-kube-api-access-6z95q\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.109769 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.109734    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8cff3a26-844e-4c57-a964-52cb78c0c894-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.109894 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.109788    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cff3a26-844e-4c57-a964-52cb78c0c894-tmp\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.182993 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.182961    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"]
Nov 04 02:00:47.211132 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.211103    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8cff3a26-844e-4c57-a964-52cb78c0c894-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.211308 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.211150    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z95q\" (UniqueName: \"kubernetes.io/projected/8cff3a26-844e-4c57-a964-52cb78c0c894-kube-api-access-6z95q\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.211308 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.211183    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8cff3a26-844e-4c57-a964-52cb78c0c894-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.211308 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.211209    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cff3a26-844e-4c57-a964-52cb78c0c894-tmp\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.211308 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.211255    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8cff3a26-844e-4c57-a964-52cb78c0c894-metrics-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.211726 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.211326    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8cff3a26-844e-4c57-a964-52cb78c0c894-manifests\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.211778 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.211723    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8cff3a26-844e-4c57-a964-52cb78c0c894-manifests\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.213462 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.213436    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cff3a26-844e-4c57-a964-52cb78c0c894-tmp\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.232432 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.232407    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 04 02:00:47.232564 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.232550    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 04 02:00:47.232606 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.232581    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 04 02:00:47.232875 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.232861    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 04 02:00:47.232980 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.232967    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-2szzx\""
Nov 04 02:00:47.236220 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.236203    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 04 02:00:47.244159 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.244135    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8cff3a26-844e-4c57-a964-52cb78c0c894-metrics-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.244159 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.244150    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8cff3a26-844e-4c57-a964-52cb78c0c894-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.244372 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.244134    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8cff3a26-844e-4c57-a964-52cb78c0c894-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.276056 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.276025    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z95q\" (UniqueName: \"kubernetes.io/projected/8cff3a26-844e-4c57-a964-52cb78c0c894-kube-api-access-6z95q\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l\" (UID: \"8cff3a26-844e-4c57-a964-52cb78c0c894\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.388577 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.388491    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:47.876930 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.876908    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"]
Nov 04 02:00:47.879658 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:00:47.879635    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cff3a26_844e_4c57_a964_52cb78c0c894.slice/crio-2d6bfd1d38b0ec4705db572d6b25ffaf68352729f11585b6aae07394660f8960 WatchSource:0}: Error finding container 2d6bfd1d38b0ec4705db572d6b25ffaf68352729f11585b6aae07394660f8960: Status 404 returned error can't find the container with id 2d6bfd1d38b0ec4705db572d6b25ffaf68352729f11585b6aae07394660f8960
Nov 04 02:00:47.881470 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:47.881455    4865 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:00:48.138152 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:48.138068    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l" event={"ID":"8cff3a26-844e-4c57-a964-52cb78c0c894","Type":"ContainerStarted","Data":"2d6bfd1d38b0ec4705db572d6b25ffaf68352729f11585b6aae07394660f8960"}
Nov 04 02:00:53.162780 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:53.162696    4865 generic.go:356] "Generic (PLEG): container finished" podID="8cff3a26-844e-4c57-a964-52cb78c0c894" containerID="9ffcc425273561861203df058abc93f2e68440108703fde226fd92ac81c6d620" exitCode=0
Nov 04 02:00:53.162780 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:53.162735    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l" event={"ID":"8cff3a26-844e-4c57-a964-52cb78c0c894","Type":"ContainerDied","Data":"9ffcc425273561861203df058abc93f2e68440108703fde226fd92ac81c6d620"}
Nov 04 02:00:54.169104 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:54.169069    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l" event={"ID":"8cff3a26-844e-4c57-a964-52cb78c0c894","Type":"ContainerStarted","Data":"f55238e19e1325d2c08f9bf5c0c1835e22b831bc14136342f2ab139e33ae1cb8"}
Nov 04 02:00:54.169562 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:54.169190    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:00:54.418371 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:00:54.418237    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l" podStartSLOduration=2.830755962 podStartE2EDuration="8.418222285s" totalImagesPullingTime="5.117411929s" totalInitContainerRuntime="470.054394ms" isStatefulPod=false podCreationTimestamp="2026-11-04 02:00:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:00:54.367505636 +0000 UTC m=+317.594292305" watchObservedRunningTime="2026-11-04 02:00:54.418222285 +0000 UTC m=+317.645008952"
Nov 04 02:00:58.403024 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:00:58.402985    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:01:05.176431 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:05.176393    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l"
Nov 04 02:01:28.403083 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:01:28.403045    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:01:37.553482 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:37.553447    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596"]
Nov 04 02:01:37.559024 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:37.559003    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596"
Nov 04 02:01:37.636433 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:37.636396    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e12e95d-d3af-4ec4-821a-e580ec557555-tmp\") pod \"kube-auth-proxy-77fbbd9bf8-td596\" (UID: \"0e12e95d-d3af-4ec4-821a-e580ec557555\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596"
Nov 04 02:01:37.636621 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:37.636464    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e12e95d-d3af-4ec4-821a-e580ec557555-tls-certs\") pod \"kube-auth-proxy-77fbbd9bf8-td596\" (UID: \"0e12e95d-d3af-4ec4-821a-e580ec557555\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596"
Nov 04 02:01:37.636621 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:37.636572    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptnbr\" (UniqueName: \"kubernetes.io/projected/0e12e95d-d3af-4ec4-821a-e580ec557555-kube-api-access-ptnbr\") pod \"kube-auth-proxy-77fbbd9bf8-td596\" (UID: \"0e12e95d-d3af-4ec4-821a-e580ec557555\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596"
Nov 04 02:01:37.682392 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:37.682356    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 04 02:01:37.682556 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:37.682399    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 04 02:01:37.682556 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:37.682468    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-krskw\""
Nov 04 02:01:37.683707 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:37.683685    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596"]
Nov 04 02:01:37.737737 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:37.737707    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e12e95d-d3af-4ec4-821a-e580ec557555-tls-certs\") pod \"kube-auth-proxy-77fbbd9bf8-td596\" (UID: \"0e12e95d-d3af-4ec4-821a-e580ec557555\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596"
Nov 04 02:01:37.737918 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:37.737750    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptnbr\" (UniqueName: \"kubernetes.io/projected/0e12e95d-d3af-4ec4-821a-e580ec557555-kube-api-access-ptnbr\") pod \"kube-auth-proxy-77fbbd9bf8-td596\" (UID: \"0e12e95d-d3af-4ec4-821a-e580ec557555\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596"
Nov 04 02:01:37.737918 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:37.737812    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e12e95d-d3af-4ec4-821a-e580ec557555-tmp\") pod \"kube-auth-proxy-77fbbd9bf8-td596\" (UID: \"0e12e95d-d3af-4ec4-821a-e580ec557555\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596"
Nov 04 02:01:37.740047 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:37.740019    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e12e95d-d3af-4ec4-821a-e580ec557555-tmp\") pod \"kube-auth-proxy-77fbbd9bf8-td596\" (UID: \"0e12e95d-d3af-4ec4-821a-e580ec557555\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596"
Nov 04 02:01:37.740299 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:37.740261    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e12e95d-d3af-4ec4-821a-e580ec557555-tls-certs\") pod \"kube-auth-proxy-77fbbd9bf8-td596\" (UID: \"0e12e95d-d3af-4ec4-821a-e580ec557555\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596"
Nov 04 02:01:37.808080 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:37.808010    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptnbr\" (UniqueName: \"kubernetes.io/projected/0e12e95d-d3af-4ec4-821a-e580ec557555-kube-api-access-ptnbr\") pod \"kube-auth-proxy-77fbbd9bf8-td596\" (UID: \"0e12e95d-d3af-4ec4-821a-e580ec557555\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596"
Nov 04 02:01:37.870192 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:37.870146    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596"
Nov 04 02:01:38.243898 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:38.243871    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596"]
Nov 04 02:01:38.245946 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:01:38.245913    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e12e95d_d3af_4ec4_821a_e580ec557555.slice/crio-ec8c215ed90e48c724b2a0410dd03848b9354a6737dd5e55205dc13a167826f1 WatchSource:0}: Error finding container ec8c215ed90e48c724b2a0410dd03848b9354a6737dd5e55205dc13a167826f1: Status 404 returned error can't find the container with id ec8c215ed90e48c724b2a0410dd03848b9354a6737dd5e55205dc13a167826f1
Nov 04 02:01:38.352659 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:38.352624    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596" event={"ID":"0e12e95d-d3af-4ec4-821a-e580ec557555","Type":"ContainerStarted","Data":"ec8c215ed90e48c724b2a0410dd03848b9354a6737dd5e55205dc13a167826f1"}
Nov 04 02:01:41.367317 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.367264    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596" event={"ID":"0e12e95d-d3af-4ec4-821a-e580ec557555","Type":"ContainerStarted","Data":"0a43c0390e1d007b8a6db090287ac650712fdb62d594161d2f63c8db4e3851b2"}
Nov 04 02:01:41.487170 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.487117    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596" podStartSLOduration=1.587947161 podStartE2EDuration="4.487103693s" totalImagesPullingTime="2.899156532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:41.486641374 +0000 UTC m=+364.713428045" watchObservedRunningTime="2026-11-04 02:01:41.487103693 +0000 UTC m=+364.713890361"
Nov 04 02:01:41.558485 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.558457    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-7sgdp"]
Nov 04 02:01:41.560437 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.560422    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:01:41.560555 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.560536    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-7sgdp"
Nov 04 02:01:41.562463 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.562441    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"
Nov 04 02:01:41.572520 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.572499    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn2b6\" (UniqueName: \"kubernetes.io/projected/26682e91-0320-4953-a6cf-a9abdca87071-kube-api-access-sn2b6\") pod \"authorino-56c6c9ddd5-7sgdp\" (UID: \"26682e91-0320-4953-a6cf-a9abdca87071\") " pod="kuadrant-system/authorino-56c6c9ddd5-7sgdp"
Nov 04 02:01:41.572615 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.572532    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqkdm\" (UniqueName: \"kubernetes.io/projected/ac4b347b-ef59-44da-a807-8dc79596178e-kube-api-access-mqkdm\") pod \"limitador-limitador-7c9b87d574-gdhhm\" (UID: \"ac4b347b-ef59-44da-a807-8dc79596178e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"
Nov 04 02:01:41.572615 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.572608    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ac4b347b-ef59-44da-a807-8dc79596178e-config-file\") pod \"limitador-limitador-7c9b87d574-gdhhm\" (UID: \"ac4b347b-ef59-44da-a807-8dc79596178e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"
Nov 04 02:01:41.617625 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.617596    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-7sgdp"]
Nov 04 02:01:41.619545 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.619430    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 04 02:01:41.619545 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.619437    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 04 02:01:41.619545 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.619514    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 04 02:01:41.621001 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.620979    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:01:41.673798 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.673763    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn2b6\" (UniqueName: \"kubernetes.io/projected/26682e91-0320-4953-a6cf-a9abdca87071-kube-api-access-sn2b6\") pod \"authorino-56c6c9ddd5-7sgdp\" (UID: \"26682e91-0320-4953-a6cf-a9abdca87071\") " pod="kuadrant-system/authorino-56c6c9ddd5-7sgdp"
Nov 04 02:01:41.673944 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.673805    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqkdm\" (UniqueName: \"kubernetes.io/projected/ac4b347b-ef59-44da-a807-8dc79596178e-kube-api-access-mqkdm\") pod \"limitador-limitador-7c9b87d574-gdhhm\" (UID: \"ac4b347b-ef59-44da-a807-8dc79596178e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"
Nov 04 02:01:41.673944 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.673831    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ac4b347b-ef59-44da-a807-8dc79596178e-config-file\") pod \"limitador-limitador-7c9b87d574-gdhhm\" (UID: \"ac4b347b-ef59-44da-a807-8dc79596178e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"
Nov 04 02:01:41.674555 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.674537    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ac4b347b-ef59-44da-a807-8dc79596178e-config-file\") pod \"limitador-limitador-7c9b87d574-gdhhm\" (UID: \"ac4b347b-ef59-44da-a807-8dc79596178e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"
Nov 04 02:01:41.682890 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.682871    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qh6x5\""
Nov 04 02:01:41.682998 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.682948    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xxnr6\""
Nov 04 02:01:41.736141 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.736112    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn2b6\" (UniqueName: \"kubernetes.io/projected/26682e91-0320-4953-a6cf-a9abdca87071-kube-api-access-sn2b6\") pod \"authorino-56c6c9ddd5-7sgdp\" (UID: \"26682e91-0320-4953-a6cf-a9abdca87071\") " pod="kuadrant-system/authorino-56c6c9ddd5-7sgdp"
Nov 04 02:01:41.736311 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.736272    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqkdm\" (UniqueName: \"kubernetes.io/projected/ac4b347b-ef59-44da-a807-8dc79596178e-kube-api-access-mqkdm\") pod \"limitador-limitador-7c9b87d574-gdhhm\" (UID: \"ac4b347b-ef59-44da-a807-8dc79596178e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"
Nov 04 02:01:41.881332 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.881247    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-7sgdp"
Nov 04 02:01:41.884236 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.884215    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:01:41.887339 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:41.887320    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"
Nov 04 02:01:42.251441 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:42.251418    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-7sgdp"]
Nov 04 02:01:42.253558 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:01:42.253530    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26682e91_0320_4953_a6cf_a9abdca87071.slice/crio-b99cba7ae5001e13e51ac670332b3f1e99c30dbc5e1c29c65e42bf0e68217e97 WatchSource:0}: Error finding container b99cba7ae5001e13e51ac670332b3f1e99c30dbc5e1c29c65e42bf0e68217e97: Status 404 returned error can't find the container with id b99cba7ae5001e13e51ac670332b3f1e99c30dbc5e1c29c65e42bf0e68217e97
Nov 04 02:01:42.303623 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:42.303602    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:01:42.305067 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:01:42.305039    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac4b347b_ef59_44da_a807_8dc79596178e.slice/crio-763339532c25af9196ab759480e8d265a57f839d7303321e30433b785cc47dfc WatchSource:0}: Error finding container 763339532c25af9196ab759480e8d265a57f839d7303321e30433b785cc47dfc: Status 404 returned error can't find the container with id 763339532c25af9196ab759480e8d265a57f839d7303321e30433b785cc47dfc
Nov 04 02:01:42.372809 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:42.372774    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-7sgdp" event={"ID":"26682e91-0320-4953-a6cf-a9abdca87071","Type":"ContainerStarted","Data":"b99cba7ae5001e13e51ac670332b3f1e99c30dbc5e1c29c65e42bf0e68217e97"}
Nov 04 02:01:42.374023 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:42.373999    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-gdhhm" event={"ID":"ac4b347b-ef59-44da-a807-8dc79596178e","Type":"ContainerStarted","Data":"763339532c25af9196ab759480e8d265a57f839d7303321e30433b785cc47dfc"}
Nov 04 02:01:46.396879 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:46.396845    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-7sgdp" event={"ID":"26682e91-0320-4953-a6cf-a9abdca87071","Type":"ContainerStarted","Data":"8522127b03ab4bf49fe2e279ba9bec85de135cb2e4bd8680d612864ceab91a5b"}
Nov 04 02:01:46.398317 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:46.398291    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-gdhhm" event={"ID":"ac4b347b-ef59-44da-a807-8dc79596178e","Type":"ContainerStarted","Data":"2ebf4b7333c422a15c1458a4dcc769f0be4ce8cdcb677c5bdd0428ec16dcc8cf"}
Nov 04 02:01:46.398457 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:46.398435    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"
Nov 04 02:01:46.513458 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:46.513366    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-7sgdp" podStartSLOduration=1.558488016 podStartE2EDuration="5.513353811s" totalImagesPullingTime="3.954865795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:46.512736938 +0000 UTC m=+369.739523599" watchObservedRunningTime="2026-11-04 02:01:46.513353811 +0000 UTC m=+369.740140475"
Nov 04 02:01:46.631044 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:46.630990    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-gdhhm" podStartSLOduration=1.67473557 podStartE2EDuration="5.630977111s" totalImagesPullingTime="3.956241541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:46.630762194 +0000 UTC m=+369.857548864" watchObservedRunningTime="2026-11-04 02:01:46.630977111 +0000 UTC m=+369.857763779"
Nov 04 02:01:57.403423 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:01:57.403391    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"
Nov 04 02:01:58.402971 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:01:58.402930    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:02:28.402137 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:02:28.402099    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:02:58.402514 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:02:58.402479    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:03:09.281794 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.281752    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"]
Nov 04 02:03:09.283993 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.283968    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.390132 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.390099    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"]
Nov 04 02:03:09.430518 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.430470    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4276cf3a-e09a-436a-9e38-a87ba1e80f30-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.430722 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.430527    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgqhd\" (UniqueName: \"kubernetes.io/projected/4276cf3a-e09a-436a-9e38-a87ba1e80f30-kube-api-access-bgqhd\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.430722 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.430636    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4276cf3a-e09a-436a-9e38-a87ba1e80f30-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.430722 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.430669    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4276cf3a-e09a-436a-9e38-a87ba1e80f30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.430722 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.430703    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4276cf3a-e09a-436a-9e38-a87ba1e80f30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.430869 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.430809    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4276cf3a-e09a-436a-9e38-a87ba1e80f30-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.430869 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.430861    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4276cf3a-e09a-436a-9e38-a87ba1e80f30-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.532084 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.531999    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4276cf3a-e09a-436a-9e38-a87ba1e80f30-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.532084 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.532038    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4276cf3a-e09a-436a-9e38-a87ba1e80f30-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.532084 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.532060    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgqhd\" (UniqueName: \"kubernetes.io/projected/4276cf3a-e09a-436a-9e38-a87ba1e80f30-kube-api-access-bgqhd\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.532396 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.532114    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4276cf3a-e09a-436a-9e38-a87ba1e80f30-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.532396 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.532145    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4276cf3a-e09a-436a-9e38-a87ba1e80f30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.532396 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.532179    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4276cf3a-e09a-436a-9e38-a87ba1e80f30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.532396 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.532224    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4276cf3a-e09a-436a-9e38-a87ba1e80f30-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.533012 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.532985    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4276cf3a-e09a-436a-9e38-a87ba1e80f30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.534515 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.534493    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4276cf3a-e09a-436a-9e38-a87ba1e80f30-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.534613 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.534525    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4276cf3a-e09a-436a-9e38-a87ba1e80f30-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.534817 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.534800    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4276cf3a-e09a-436a-9e38-a87ba1e80f30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.534874 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.534836    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4276cf3a-e09a-436a-9e38-a87ba1e80f30-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.594963 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.594935    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4276cf3a-e09a-436a-9e38-a87ba1e80f30-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.595152 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.595085    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgqhd\" (UniqueName: \"kubernetes.io/projected/4276cf3a-e09a-436a-9e38-a87ba1e80f30-kube-api-access-bgqhd\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8pm7p\" (UID: \"4276cf3a-e09a-436a-9e38-a87ba1e80f30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:09.894519 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:09.894429    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:10.320836 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:10.320808    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"]
Nov 04 02:03:10.323419 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:03:10.323391    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4276cf3a_e09a_436a_9e38_a87ba1e80f30.slice/crio-bec3577bfc8123b1b8237fbfa166f00f77f25a1f3f550a21cf0c7c0772be020a WatchSource:0}: Error finding container bec3577bfc8123b1b8237fbfa166f00f77f25a1f3f550a21cf0c7c0772be020a: Status 404 returned error can't find the container with id bec3577bfc8123b1b8237fbfa166f00f77f25a1f3f550a21cf0c7c0772be020a
Nov 04 02:03:10.325452 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:10.325419    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:03:10.325525 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:10.325483    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:03:10.751788 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:10.751747    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p" event={"ID":"4276cf3a-e09a-436a-9e38-a87ba1e80f30","Type":"ContainerStarted","Data":"99f8e57d3bf49548852b4949d410b1cb2c539739aa648c1dad4b3bf502f2d3e4"}
Nov 04 02:03:10.751991 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:10.751809    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p" event={"ID":"4276cf3a-e09a-436a-9e38-a87ba1e80f30","Type":"ContainerStarted","Data":"bec3577bfc8123b1b8237fbfa166f00f77f25a1f3f550a21cf0c7c0772be020a"}
Nov 04 02:03:10.751991 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:10.751961    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:11.007419 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:11.007310    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p" podStartSLOduration=2.007293359 podStartE2EDuration="2.007293359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:03:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:03:10.955374143 +0000 UTC m=+454.182160811" watchObservedRunningTime="2026-11-04 02:03:11.007293359 +0000 UTC m=+454.234080017"
Nov 04 02:03:11.758028 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:11.757991    4865 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-8pm7p container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 02:03:11.758431 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:11.758072    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p" podUID="4276cf3a-e09a-436a-9e38-a87ba1e80f30" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 02:03:14.756952 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:14.756920    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p"
Nov 04 02:03:14.985692 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:14.985645    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"]
Nov 04 02:03:14.985922 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:14.985891    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp" podUID="73c19691-9fba-462f-9b0b-66b093d8f3a2" containerName="discovery" containerID="cri-o://9b181a0527d810925713e4a3f0d2208ca8ff7207677b29141a6f37a7178078d7" gracePeriod=30
Nov 04 02:03:15.231804 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.231775    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 02:03:15.385535 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.385457    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73c19691-9fba-462f-9b0b-66b093d8f3a2-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/73c19691-9fba-462f-9b0b-66b093d8f3a2-local-certs\") pod \"73c19691-9fba-462f-9b0b-66b093d8f3a2\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") "
Nov 04 02:03:15.385535 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.385506    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-cacerts\" (UniqueName: \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-cacerts\") pod \"73c19691-9fba-462f-9b0b-66b093d8f3a2\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") "
Nov 04 02:03:15.385778 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.385547    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-kubeconfig\") pod \"73c19691-9fba-462f-9b0b-66b093d8f3a2\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") "
Nov 04 02:03:15.385778 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.385596    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73c19691-9fba-462f-9b0b-66b093d8f3a2-kube-api-access-9bb7c\" (UniqueName: \"kubernetes.io/projected/73c19691-9fba-462f-9b0b-66b093d8f3a2-kube-api-access-9bb7c\") pod \"73c19691-9fba-462f-9b0b-66b093d8f3a2\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") "
Nov 04 02:03:15.385778 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.385637    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-token\" (UniqueName: \"kubernetes.io/projected/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-token\") pod \"73c19691-9fba-462f-9b0b-66b093d8f3a2\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") "
Nov 04 02:03:15.385778 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.385685    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-csr-dns-cert\") pod \"73c19691-9fba-462f-9b0b-66b093d8f3a2\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") "
Nov 04 02:03:15.385778 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.385739    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-csr-ca-configmap\") pod \"73c19691-9fba-462f-9b0b-66b093d8f3a2\" (UID: \"73c19691-9fba-462f-9b0b-66b093d8f3a2\") "
Nov 04 02:03:15.386379 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.386324    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-csr-ca-configmap" pod "73c19691-9fba-462f-9b0b-66b093d8f3a2" (UID: "73c19691-9fba-462f-9b0b-66b093d8f3a2"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:03:15.388542 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.388500    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-cacerts" pod "73c19691-9fba-462f-9b0b-66b093d8f3a2" (UID: "73c19691-9fba-462f-9b0b-66b093d8f3a2"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:03:15.388649 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.388568    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-token" pod "73c19691-9fba-462f-9b0b-66b093d8f3a2" (UID: "73c19691-9fba-462f-9b0b-66b093d8f3a2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:03:15.388649 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.388579    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73c19691-9fba-462f-9b0b-66b093d8f3a2-local-certs" pod "73c19691-9fba-462f-9b0b-66b093d8f3a2" (UID: "73c19691-9fba-462f-9b0b-66b093d8f3a2"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:03:15.388649 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.388591    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-csr-dns-cert" pod "73c19691-9fba-462f-9b0b-66b093d8f3a2" (UID: "73c19691-9fba-462f-9b0b-66b093d8f3a2"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:03:15.388649 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.388569    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73c19691-9fba-462f-9b0b-66b093d8f3a2-kube-api-access-9bb7c" pod "73c19691-9fba-462f-9b0b-66b093d8f3a2" (UID: "73c19691-9fba-462f-9b0b-66b093d8f3a2"). InnerVolumeSpecName "kube-api-access-9bb7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:03:15.388920 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.388903    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-kubeconfig" pod "73c19691-9fba-462f-9b0b-66b093d8f3a2" (UID: "73c19691-9fba-462f-9b0b-66b093d8f3a2"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:03:15.486651 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.486613    4865 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-cacerts\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:03:15.486651 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.486645    4865 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-kubeconfig\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:03:15.486651 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.486660    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bb7c\" (UniqueName: \"kubernetes.io/projected/73c19691-9fba-462f-9b0b-66b093d8f3a2-kube-api-access-9bb7c\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:03:15.486886 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.486670    4865 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:03:15.486886 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.486679    4865 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-csr-dns-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:03:15.486886 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.486687    4865 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/73c19691-9fba-462f-9b0b-66b093d8f3a2-istio-csr-ca-configmap\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:03:15.486886 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.486698    4865 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/73c19691-9fba-462f-9b0b-66b093d8f3a2-local-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:03:15.594395 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.594357    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-l8pbh"]
Nov 04 02:03:15.594690 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.594678    4865 cpu_manager.go:414] "containerMap: removing container" podUID="73c19691-9fba-462f-9b0b-66b093d8f3a2" containerName="discovery"
Nov 04 02:03:15.594732 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.594691    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73c19691-9fba-462f-9b0b-66b093d8f3a2" containerName="discovery"
Nov 04 02:03:15.594781 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.594772    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="73c19691-9fba-462f-9b0b-66b093d8f3a2" containerName="discovery"
Nov 04 02:03:15.597036 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.597010    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-l8pbh"
Nov 04 02:03:15.654860 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.654775    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 04 02:03:15.655022 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.654908    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-2m5hr\""
Nov 04 02:03:15.655022 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.654933    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Nov 04 02:03:15.655022 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.654994    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 04 02:03:15.655150 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.655061    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Nov 04 02:03:15.656688 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.656666    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-l8pbh"]
Nov 04 02:03:15.688121 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.688089    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9e9997e-029b-4d43-a61e-c54bed553dfe-metrics-certs\") pod \"odh-model-controller-769875db56-l8pbh\" (UID: \"e9e9997e-029b-4d43-a61e-c54bed553dfe\") " pod="opendatahub/odh-model-controller-769875db56-l8pbh"
Nov 04 02:03:15.688271 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.688134    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e9e9997e-029b-4d43-a61e-c54bed553dfe-cert\") pod \"odh-model-controller-769875db56-l8pbh\" (UID: \"e9e9997e-029b-4d43-a61e-c54bed553dfe\") " pod="opendatahub/odh-model-controller-769875db56-l8pbh"
Nov 04 02:03:15.688271 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.688159    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhtrk\" (UniqueName: \"kubernetes.io/projected/e9e9997e-029b-4d43-a61e-c54bed553dfe-kube-api-access-bhtrk\") pod \"odh-model-controller-769875db56-l8pbh\" (UID: \"e9e9997e-029b-4d43-a61e-c54bed553dfe\") " pod="opendatahub/odh-model-controller-769875db56-l8pbh"
Nov 04 02:03:15.781668 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.781627    4865 generic.go:356] "Generic (PLEG): container finished" podID="73c19691-9fba-462f-9b0b-66b093d8f3a2" containerID="9b181a0527d810925713e4a3f0d2208ca8ff7207677b29141a6f37a7178078d7" exitCode=0
Nov 04 02:03:15.782066 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.781677    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"
Nov 04 02:03:15.782066 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.781690    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp" event={"ID":"73c19691-9fba-462f-9b0b-66b093d8f3a2","Type":"ContainerDied","Data":"9b181a0527d810925713e4a3f0d2208ca8ff7207677b29141a6f37a7178078d7"}
Nov 04 02:03:15.782066 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.781739    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp" event={"ID":"73c19691-9fba-462f-9b0b-66b093d8f3a2","Type":"ContainerDied","Data":"f4ca6ec906326cd5ac4e130ccc0efbe929367b33ec67366e91e76e4928a19c18"}
Nov 04 02:03:15.782066 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.781766    4865 scope.go:122] "RemoveContainer" containerID="9b181a0527d810925713e4a3f0d2208ca8ff7207677b29141a6f37a7178078d7"
Nov 04 02:03:15.788798 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.788770    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e9e9997e-029b-4d43-a61e-c54bed553dfe-cert\") pod \"odh-model-controller-769875db56-l8pbh\" (UID: \"e9e9997e-029b-4d43-a61e-c54bed553dfe\") " pod="opendatahub/odh-model-controller-769875db56-l8pbh"
Nov 04 02:03:15.788905 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.788819    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhtrk\" (UniqueName: \"kubernetes.io/projected/e9e9997e-029b-4d43-a61e-c54bed553dfe-kube-api-access-bhtrk\") pod \"odh-model-controller-769875db56-l8pbh\" (UID: \"e9e9997e-029b-4d43-a61e-c54bed553dfe\") " pod="opendatahub/odh-model-controller-769875db56-l8pbh"
Nov 04 02:03:15.789132 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.788908    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9e9997e-029b-4d43-a61e-c54bed553dfe-metrics-certs\") pod \"odh-model-controller-769875db56-l8pbh\" (UID: \"e9e9997e-029b-4d43-a61e-c54bed553dfe\") " pod="opendatahub/odh-model-controller-769875db56-l8pbh"
Nov 04 02:03:15.790652 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.790633    4865 scope.go:122] "RemoveContainer" containerID="9b181a0527d810925713e4a3f0d2208ca8ff7207677b29141a6f37a7178078d7"
Nov 04 02:03:15.790960 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:03:15.790936    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b181a0527d810925713e4a3f0d2208ca8ff7207677b29141a6f37a7178078d7\": container with ID starting with 9b181a0527d810925713e4a3f0d2208ca8ff7207677b29141a6f37a7178078d7 not found: ID does not exist" containerID="9b181a0527d810925713e4a3f0d2208ca8ff7207677b29141a6f37a7178078d7"
Nov 04 02:03:15.791035 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.790971    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b181a0527d810925713e4a3f0d2208ca8ff7207677b29141a6f37a7178078d7"} err="failed to get container status \"9b181a0527d810925713e4a3f0d2208ca8ff7207677b29141a6f37a7178078d7\": rpc error: code = NotFound desc = could not find container \"9b181a0527d810925713e4a3f0d2208ca8ff7207677b29141a6f37a7178078d7\": container with ID starting with 9b181a0527d810925713e4a3f0d2208ca8ff7207677b29141a6f37a7178078d7 not found: ID does not exist"
Nov 04 02:03:15.791370 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.791351    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9e9997e-029b-4d43-a61e-c54bed553dfe-metrics-certs\") pod \"odh-model-controller-769875db56-l8pbh\" (UID: \"e9e9997e-029b-4d43-a61e-c54bed553dfe\") " pod="opendatahub/odh-model-controller-769875db56-l8pbh"
Nov 04 02:03:15.791446 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.791407    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e9e9997e-029b-4d43-a61e-c54bed553dfe-cert\") pod \"odh-model-controller-769875db56-l8pbh\" (UID: \"e9e9997e-029b-4d43-a61e-c54bed553dfe\") " pod="opendatahub/odh-model-controller-769875db56-l8pbh"
Nov 04 02:03:15.851201 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.851165    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhtrk\" (UniqueName: \"kubernetes.io/projected/e9e9997e-029b-4d43-a61e-c54bed553dfe-kube-api-access-bhtrk\") pod \"odh-model-controller-769875db56-l8pbh\" (UID: \"e9e9997e-029b-4d43-a61e-c54bed553dfe\") " pod="opendatahub/odh-model-controller-769875db56-l8pbh"
Nov 04 02:03:15.909346 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.909255    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-l8pbh"
Nov 04 02:03:15.952047 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.952014    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"]
Nov 04 02:03:15.956618 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:15.956591    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp"]
Nov 04 02:03:16.280843 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:16.280811    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-l8pbh"]
Nov 04 02:03:16.284072 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:03:16.284046    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9e9997e_029b_4d43_a61e_c54bed553dfe.slice/crio-227ece09f7f3b94f3d7d954234a50ab7d66c23f736f04fe1a9ab34afadb79231 WatchSource:0}: Error finding container 227ece09f7f3b94f3d7d954234a50ab7d66c23f736f04fe1a9ab34afadb79231: Status 404 returned error can't find the container with id 227ece09f7f3b94f3d7d954234a50ab7d66c23f736f04fe1a9ab34afadb79231
Nov 04 02:03:16.788123 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:16.788089    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-l8pbh" event={"ID":"e9e9997e-029b-4d43-a61e-c54bed553dfe","Type":"ContainerStarted","Data":"227ece09f7f3b94f3d7d954234a50ab7d66c23f736f04fe1a9ab34afadb79231"}
Nov 04 02:03:16.893610 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:16.893579    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73c19691-9fba-462f-9b0b-66b093d8f3a2" path="/var/lib/kubelet/pods/73c19691-9fba-462f-9b0b-66b093d8f3a2/volumes"
Nov 04 02:03:19.803093 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:19.803049    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-l8pbh" event={"ID":"e9e9997e-029b-4d43-a61e-c54bed553dfe","Type":"ContainerStarted","Data":"da963fd1040039ca10b86c269a1ad841746721f355e4abbbceb15a4e1deaa998"}
Nov 04 02:03:19.803581 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:19.803122    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-l8pbh"
Nov 04 02:03:19.922545 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:19.922442    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-l8pbh" podStartSLOduration=1.629334977 podStartE2EDuration="4.922427416s" totalImagesPullingTime="3.293092439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:03:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:03:19.921386087 +0000 UTC m=+463.148172775" watchObservedRunningTime="2026-11-04 02:03:19.922427416 +0000 UTC m=+463.149214081"
Nov 04 02:03:20.354808 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.354770    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"]
Nov 04 02:03:20.358119 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.358095    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:20.462039 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.462004    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 02:03:20.462039 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.462032    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 04 02:03:20.462246 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.462045    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 02:03:20.462246 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.462019    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hnsgt\""
Nov 04 02:03:20.462246 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.462072    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 04 02:03:20.463492 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.463472    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"]
Nov 04 02:03:20.532107 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.532074    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55334824-2552-453a-a537-a6a916468bef-metrics-certs\") pod \"maas-api-6c69c947c-bf8hh\" (UID: \"55334824-2552-453a-a537-a6a916468bef\") " pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:20.532276 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.532118    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55334824-2552-453a-a537-a6a916468bef-maas-api-tls\") pod \"maas-api-6c69c947c-bf8hh\" (UID: \"55334824-2552-453a-a537-a6a916468bef\") " pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:20.532276 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.532221    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55334824-2552-453a-a537-a6a916468bef-openshift-service-ca\") pod \"maas-api-6c69c947c-bf8hh\" (UID: \"55334824-2552-453a-a537-a6a916468bef\") " pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:20.532362 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.532332    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6wg2\" (UniqueName: \"kubernetes.io/projected/55334824-2552-453a-a537-a6a916468bef-kube-api-access-t6wg2\") pod \"maas-api-6c69c947c-bf8hh\" (UID: \"55334824-2552-453a-a537-a6a916468bef\") " pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:20.633408 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.633330    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6wg2\" (UniqueName: \"kubernetes.io/projected/55334824-2552-453a-a537-a6a916468bef-kube-api-access-t6wg2\") pod \"maas-api-6c69c947c-bf8hh\" (UID: \"55334824-2552-453a-a537-a6a916468bef\") " pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:20.633408 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.633387    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55334824-2552-453a-a537-a6a916468bef-metrics-certs\") pod \"maas-api-6c69c947c-bf8hh\" (UID: \"55334824-2552-453a-a537-a6a916468bef\") " pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:20.633613 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.633427    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55334824-2552-453a-a537-a6a916468bef-maas-api-tls\") pod \"maas-api-6c69c947c-bf8hh\" (UID: \"55334824-2552-453a-a537-a6a916468bef\") " pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:20.633613 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.633476    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55334824-2552-453a-a537-a6a916468bef-openshift-service-ca\") pod \"maas-api-6c69c947c-bf8hh\" (UID: \"55334824-2552-453a-a537-a6a916468bef\") " pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:20.634198 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.634168    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55334824-2552-453a-a537-a6a916468bef-openshift-service-ca\") pod \"maas-api-6c69c947c-bf8hh\" (UID: \"55334824-2552-453a-a537-a6a916468bef\") " pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:20.635924 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.635901    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55334824-2552-453a-a537-a6a916468bef-metrics-certs\") pod \"maas-api-6c69c947c-bf8hh\" (UID: \"55334824-2552-453a-a537-a6a916468bef\") " pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:20.636020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.635968    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55334824-2552-453a-a537-a6a916468bef-maas-api-tls\") pod \"maas-api-6c69c947c-bf8hh\" (UID: \"55334824-2552-453a-a537-a6a916468bef\") " pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:20.695613 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.695572    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6wg2\" (UniqueName: \"kubernetes.io/projected/55334824-2552-453a-a537-a6a916468bef-kube-api-access-t6wg2\") pod \"maas-api-6c69c947c-bf8hh\" (UID: \"55334824-2552-453a-a537-a6a916468bef\") " pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:20.970379 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:20.970339    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:21.347149 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:21.347068    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"]
Nov 04 02:03:21.349949 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:03:21.349918    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55334824_2552_453a_a537_a6a916468bef.slice/crio-bd27bbe0e6ac743fde66e3805d95e4b2f4c200b2071e9ec6a7f48c4d1cf02e95 WatchSource:0}: Error finding container bd27bbe0e6ac743fde66e3805d95e4b2f4c200b2071e9ec6a7f48c4d1cf02e95: Status 404 returned error can't find the container with id bd27bbe0e6ac743fde66e3805d95e4b2f4c200b2071e9ec6a7f48c4d1cf02e95
Nov 04 02:03:21.813631 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:21.813598    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh" event={"ID":"55334824-2552-453a-a537-a6a916468bef","Type":"ContainerStarted","Data":"bd27bbe0e6ac743fde66e3805d95e4b2f4c200b2071e9ec6a7f48c4d1cf02e95"}
Nov 04 02:03:26.003255 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:26.003211    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:03:28.403592 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:03:28.403556    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:03:28.850184 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:28.850145    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh" event={"ID":"55334824-2552-453a-a537-a6a916468bef","Type":"ContainerStarted","Data":"f36da07cebb2e449e4c0a887b8356ad1262759b6778de9403bcddc5c5acc0a91"}
Nov 04 02:03:28.850402 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:28.850274    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:28.970232 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:28.970180    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh" podStartSLOduration=2.117628536 podStartE2EDuration="8.97016748s" totalImagesPullingTime="6.852538944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:03:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:03:28.969171577 +0000 UTC m=+472.195958248" watchObservedRunningTime="2026-11-04 02:03:28.97016748 +0000 UTC m=+472.196954148"
Nov 04 02:03:30.810931 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:30.810891    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-l8pbh"
Nov 04 02:03:34.859532 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:34.859503    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:35.766290 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:35.766247    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-pt749"]
Nov 04 02:03:35.769792 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:35.769765    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-pt749"
Nov 04 02:03:35.826521 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:35.826495    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-pt749"]
Nov 04 02:03:35.857955 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:35.857925    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw4fp\" (UniqueName: \"kubernetes.io/projected/d41c1767-9cf0-4a75-b434-b77b6a96d7b3-kube-api-access-kw4fp\") pod \"authorino-6f6944bcdd-pt749\" (UID: \"d41c1767-9cf0-4a75-b434-b77b6a96d7b3\") " pod="kuadrant-system/authorino-6f6944bcdd-pt749"
Nov 04 02:03:35.959090 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:35.959051    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw4fp\" (UniqueName: \"kubernetes.io/projected/d41c1767-9cf0-4a75-b434-b77b6a96d7b3-kube-api-access-kw4fp\") pod \"authorino-6f6944bcdd-pt749\" (UID: \"d41c1767-9cf0-4a75-b434-b77b6a96d7b3\") " pod="kuadrant-system/authorino-6f6944bcdd-pt749"
Nov 04 02:03:36.021327 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:36.021230    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw4fp\" (UniqueName: \"kubernetes.io/projected/d41c1767-9cf0-4a75-b434-b77b6a96d7b3-kube-api-access-kw4fp\") pod \"authorino-6f6944bcdd-pt749\" (UID: \"d41c1767-9cf0-4a75-b434-b77b6a96d7b3\") " pod="kuadrant-system/authorino-6f6944bcdd-pt749"
Nov 04 02:03:36.081535 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:36.081502    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-pt749"
Nov 04 02:03:36.176436 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:36.176404    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-pt749"]
Nov 04 02:03:36.454357 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:36.454331    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-pt749"]
Nov 04 02:03:36.457063 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:03:36.457035    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd41c1767_9cf0_4a75_b434_b77b6a96d7b3.slice/crio-f2704d1c1ad5db2874638c93cb51649172030e1344d7a5d052ae9fcf52161853 WatchSource:0}: Error finding container f2704d1c1ad5db2874638c93cb51649172030e1344d7a5d052ae9fcf52161853: Status 404 returned error can't find the container with id f2704d1c1ad5db2874638c93cb51649172030e1344d7a5d052ae9fcf52161853
Nov 04 02:03:36.630068 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:36.630032    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-bddb76fbc-vtbff"]
Nov 04 02:03:36.634562 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:36.634540    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bddb76fbc-vtbff"
Nov 04 02:03:36.664521 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:36.664490    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjbg8\" (UniqueName: \"kubernetes.io/projected/cc66e1cb-8674-40ab-827a-d90415dc4dc5-kube-api-access-pjbg8\") pod \"authorino-bddb76fbc-vtbff\" (UID: \"cc66e1cb-8674-40ab-827a-d90415dc4dc5\") " pod="kuadrant-system/authorino-bddb76fbc-vtbff"
Nov 04 02:03:36.664648 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:36.664542    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cc66e1cb-8674-40ab-827a-d90415dc4dc5-tls-cert\") pod \"authorino-bddb76fbc-vtbff\" (UID: \"cc66e1cb-8674-40ab-827a-d90415dc4dc5\") " pod="kuadrant-system/authorino-bddb76fbc-vtbff"
Nov 04 02:03:36.690071 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:36.690042    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bddb76fbc-vtbff"]
Nov 04 02:03:36.739442 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:36.739356    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Nov 04 02:03:36.765574 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:36.765544    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjbg8\" (UniqueName: \"kubernetes.io/projected/cc66e1cb-8674-40ab-827a-d90415dc4dc5-kube-api-access-pjbg8\") pod \"authorino-bddb76fbc-vtbff\" (UID: \"cc66e1cb-8674-40ab-827a-d90415dc4dc5\") " pod="kuadrant-system/authorino-bddb76fbc-vtbff"
Nov 04 02:03:36.765718 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:36.765589    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cc66e1cb-8674-40ab-827a-d90415dc4dc5-tls-cert\") pod \"authorino-bddb76fbc-vtbff\" (UID: \"cc66e1cb-8674-40ab-827a-d90415dc4dc5\") " pod="kuadrant-system/authorino-bddb76fbc-vtbff"
Nov 04 02:03:36.768091 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:36.768070    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cc66e1cb-8674-40ab-827a-d90415dc4dc5-tls-cert\") pod \"authorino-bddb76fbc-vtbff\" (UID: \"cc66e1cb-8674-40ab-827a-d90415dc4dc5\") " pod="kuadrant-system/authorino-bddb76fbc-vtbff"
Nov 04 02:03:36.827546 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:36.827518    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjbg8\" (UniqueName: \"kubernetes.io/projected/cc66e1cb-8674-40ab-827a-d90415dc4dc5-kube-api-access-pjbg8\") pod \"authorino-bddb76fbc-vtbff\" (UID: \"cc66e1cb-8674-40ab-827a-d90415dc4dc5\") " pod="kuadrant-system/authorino-bddb76fbc-vtbff"
Nov 04 02:03:36.884264 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:36.884230    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-pt749" event={"ID":"d41c1767-9cf0-4a75-b434-b77b6a96d7b3","Type":"ContainerStarted","Data":"f2704d1c1ad5db2874638c93cb51649172030e1344d7a5d052ae9fcf52161853"}
Nov 04 02:03:36.945605 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:36.945577    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bddb76fbc-vtbff"
Nov 04 02:03:37.318582 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:37.318557    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bddb76fbc-vtbff"]
Nov 04 02:03:37.320651 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:03:37.320621    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc66e1cb_8674_40ab_827a_d90415dc4dc5.slice/crio-bdaab3a05dfb87e95e494b0c57f22635982d66ca3723c605465471f596c75951 WatchSource:0}: Error finding container bdaab3a05dfb87e95e494b0c57f22635982d66ca3723c605465471f596c75951: Status 404 returned error can't find the container with id bdaab3a05dfb87e95e494b0c57f22635982d66ca3723c605465471f596c75951
Nov 04 02:03:37.889801 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:37.889759    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bddb76fbc-vtbff" event={"ID":"cc66e1cb-8674-40ab-827a-d90415dc4dc5","Type":"ContainerStarted","Data":"bdaab3a05dfb87e95e494b0c57f22635982d66ca3723c605465471f596c75951"}
Nov 04 02:03:38.904363 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:38.904244    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-pt749" event={"ID":"d41c1767-9cf0-4a75-b434-b77b6a96d7b3","Type":"ContainerStarted","Data":"7ed8b770906de378f4c3ce31748a2444420595e1fb42acdfd5acd3674c1b8efd"}
Nov 04 02:03:38.904363 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:38.904308    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-6f6944bcdd-pt749" podUID="d41c1767-9cf0-4a75-b434-b77b6a96d7b3" containerName="authorino" containerID="cri-o://7ed8b770906de378f4c3ce31748a2444420595e1fb42acdfd5acd3674c1b8efd" gracePeriod=30
Nov 04 02:03:38.906230 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:38.906206    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bddb76fbc-vtbff" event={"ID":"cc66e1cb-8674-40ab-827a-d90415dc4dc5","Type":"ContainerStarted","Data":"0833400d93a8cf3056baf87d68ef7777418f0278972ae335ebe40fd2146a7ebc"}
Nov 04 02:03:39.021406 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.021354    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-6f6944bcdd-pt749" podStartSLOduration=2.53308938 podStartE2EDuration="4.021340208s" totalImagesPullingTime="1.488250828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:03:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:03:39.019707247 +0000 UTC m=+482.246493916" watchObservedRunningTime="2026-11-04 02:03:39.021340208 +0000 UTC m=+482.248126876"
Nov 04 02:03:39.138560 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.138493    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-bddb76fbc-vtbff" podStartSLOduration=1.998953437 podStartE2EDuration="3.138477619s" totalImagesPullingTime="1.139524182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:03:39.136138305 +0000 UTC m=+482.362924973" watchObservedRunningTime="2026-11-04 02:03:39.138477619 +0000 UTC m=+482.365264289"
Nov 04 02:03:39.166751 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.166725    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-7sgdp"]
Nov 04 02:03:39.166986 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.166962    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-7sgdp" podUID="26682e91-0320-4953-a6cf-a9abdca87071" containerName="authorino" containerID="cri-o://8522127b03ab4bf49fe2e279ba9bec85de135cb2e4bd8680d612864ceab91a5b" gracePeriod=30
Nov 04 02:03:39.178483 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.178449    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-pt749"
Nov 04 02:03:39.288828 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.288799    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d41c1767-9cf0-4a75-b434-b77b6a96d7b3-kube-api-access-kw4fp\" (UniqueName: \"kubernetes.io/projected/d41c1767-9cf0-4a75-b434-b77b6a96d7b3-kube-api-access-kw4fp\") pod \"d41c1767-9cf0-4a75-b434-b77b6a96d7b3\" (UID: \"d41c1767-9cf0-4a75-b434-b77b6a96d7b3\") "
Nov 04 02:03:39.291537 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.291499    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d41c1767-9cf0-4a75-b434-b77b6a96d7b3-kube-api-access-kw4fp" pod "d41c1767-9cf0-4a75-b434-b77b6a96d7b3" (UID: "d41c1767-9cf0-4a75-b434-b77b6a96d7b3"). InnerVolumeSpecName "kube-api-access-kw4fp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:03:39.389635 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.389598    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kw4fp\" (UniqueName: \"kubernetes.io/projected/d41c1767-9cf0-4a75-b434-b77b6a96d7b3-kube-api-access-kw4fp\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:03:39.406796 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.406738    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-7sgdp"
Nov 04 02:03:39.490756 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.490723    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26682e91-0320-4953-a6cf-a9abdca87071-kube-api-access-sn2b6\" (UniqueName: \"kubernetes.io/projected/26682e91-0320-4953-a6cf-a9abdca87071-kube-api-access-sn2b6\") pod \"26682e91-0320-4953-a6cf-a9abdca87071\" (UID: \"26682e91-0320-4953-a6cf-a9abdca87071\") "
Nov 04 02:03:39.492783 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.492758    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26682e91-0320-4953-a6cf-a9abdca87071-kube-api-access-sn2b6" pod "26682e91-0320-4953-a6cf-a9abdca87071" (UID: "26682e91-0320-4953-a6cf-a9abdca87071"). InnerVolumeSpecName "kube-api-access-sn2b6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:03:39.592307 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.592248    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn2b6\" (UniqueName: \"kubernetes.io/projected/26682e91-0320-4953-a6cf-a9abdca87071-kube-api-access-sn2b6\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:03:39.911688 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.911654    4865 generic.go:356] "Generic (PLEG): container finished" podID="d41c1767-9cf0-4a75-b434-b77b6a96d7b3" containerID="7ed8b770906de378f4c3ce31748a2444420595e1fb42acdfd5acd3674c1b8efd" exitCode=0
Nov 04 02:03:39.912138 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.911695    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-pt749"
Nov 04 02:03:39.912138 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.911736    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-pt749" event={"ID":"d41c1767-9cf0-4a75-b434-b77b6a96d7b3","Type":"ContainerDied","Data":"7ed8b770906de378f4c3ce31748a2444420595e1fb42acdfd5acd3674c1b8efd"}
Nov 04 02:03:39.912138 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.911781    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-pt749" event={"ID":"d41c1767-9cf0-4a75-b434-b77b6a96d7b3","Type":"ContainerDied","Data":"f2704d1c1ad5db2874638c93cb51649172030e1344d7a5d052ae9fcf52161853"}
Nov 04 02:03:39.912138 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.911805    4865 scope.go:122] "RemoveContainer" containerID="7ed8b770906de378f4c3ce31748a2444420595e1fb42acdfd5acd3674c1b8efd"
Nov 04 02:03:39.913128 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.913055    4865 generic.go:356] "Generic (PLEG): container finished" podID="26682e91-0320-4953-a6cf-a9abdca87071" containerID="8522127b03ab4bf49fe2e279ba9bec85de135cb2e4bd8680d612864ceab91a5b" exitCode=0
Nov 04 02:03:39.913128 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.913084    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-7sgdp" event={"ID":"26682e91-0320-4953-a6cf-a9abdca87071","Type":"ContainerDied","Data":"8522127b03ab4bf49fe2e279ba9bec85de135cb2e4bd8680d612864ceab91a5b"}
Nov 04 02:03:39.913128 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.913099    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-7sgdp"
Nov 04 02:03:39.913128 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.913121    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-7sgdp" event={"ID":"26682e91-0320-4953-a6cf-a9abdca87071","Type":"ContainerDied","Data":"b99cba7ae5001e13e51ac670332b3f1e99c30dbc5e1c29c65e42bf0e68217e97"}
Nov 04 02:03:39.922590 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.922570    4865 scope.go:122] "RemoveContainer" containerID="7ed8b770906de378f4c3ce31748a2444420595e1fb42acdfd5acd3674c1b8efd"
Nov 04 02:03:39.922840 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:03:39.922819    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ed8b770906de378f4c3ce31748a2444420595e1fb42acdfd5acd3674c1b8efd\": container with ID starting with 7ed8b770906de378f4c3ce31748a2444420595e1fb42acdfd5acd3674c1b8efd not found: ID does not exist" containerID="7ed8b770906de378f4c3ce31748a2444420595e1fb42acdfd5acd3674c1b8efd"
Nov 04 02:03:39.922934 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.922845    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ed8b770906de378f4c3ce31748a2444420595e1fb42acdfd5acd3674c1b8efd"} err="failed to get container status \"7ed8b770906de378f4c3ce31748a2444420595e1fb42acdfd5acd3674c1b8efd\": rpc error: code = NotFound desc = could not find container \"7ed8b770906de378f4c3ce31748a2444420595e1fb42acdfd5acd3674c1b8efd\": container with ID starting with 7ed8b770906de378f4c3ce31748a2444420595e1fb42acdfd5acd3674c1b8efd not found: ID does not exist"
Nov 04 02:03:39.922934 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.922860    4865 scope.go:122] "RemoveContainer" containerID="8522127b03ab4bf49fe2e279ba9bec85de135cb2e4bd8680d612864ceab91a5b"
Nov 04 02:03:39.930139 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.930119    4865 scope.go:122] "RemoveContainer" containerID="8522127b03ab4bf49fe2e279ba9bec85de135cb2e4bd8680d612864ceab91a5b"
Nov 04 02:03:39.930486 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:03:39.930468    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8522127b03ab4bf49fe2e279ba9bec85de135cb2e4bd8680d612864ceab91a5b\": container with ID starting with 8522127b03ab4bf49fe2e279ba9bec85de135cb2e4bd8680d612864ceab91a5b not found: ID does not exist" containerID="8522127b03ab4bf49fe2e279ba9bec85de135cb2e4bd8680d612864ceab91a5b"
Nov 04 02:03:39.930551 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:39.930491    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8522127b03ab4bf49fe2e279ba9bec85de135cb2e4bd8680d612864ceab91a5b"} err="failed to get container status \"8522127b03ab4bf49fe2e279ba9bec85de135cb2e4bd8680d612864ceab91a5b\": rpc error: code = NotFound desc = could not find container \"8522127b03ab4bf49fe2e279ba9bec85de135cb2e4bd8680d612864ceab91a5b\": container with ID starting with 8522127b03ab4bf49fe2e279ba9bec85de135cb2e4bd8680d612864ceab91a5b not found: ID does not exist"
Nov 04 02:03:40.077498 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:40.077454    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-pt749"]
Nov 04 02:03:40.079322 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:40.079296    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-pt749"]
Nov 04 02:03:40.189498 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:40.189425    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-7sgdp"]
Nov 04 02:03:40.195552 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:40.195527    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-7sgdp"]
Nov 04 02:03:40.893333 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:40.893304    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="26682e91-0320-4953-a6cf-a9abdca87071" path="/var/lib/kubelet/pods/26682e91-0320-4953-a6cf-a9abdca87071/volumes"
Nov 04 02:03:40.893619 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:40.893607    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d41c1767-9cf0-4a75-b434-b77b6a96d7b3" path="/var/lib/kubelet/pods/d41c1767-9cf0-4a75-b434-b77b6a96d7b3/volumes"
Nov 04 02:03:47.482819 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.482729    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"]
Nov 04 02:03:47.483273 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.483010    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh" podUID="55334824-2552-453a-a537-a6a916468bef" containerName="maas-api" containerID="cri-o://f36da07cebb2e449e4c0a887b8356ad1262759b6778de9403bcddc5c5acc0a91" gracePeriod=30
Nov 04 02:03:47.722206 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.722179    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:47.762292 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.762200    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55334824-2552-453a-a537-a6a916468bef-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55334824-2552-453a-a537-a6a916468bef-openshift-service-ca\") pod \"55334824-2552-453a-a537-a6a916468bef\" (UID: \"55334824-2552-453a-a537-a6a916468bef\") "
Nov 04 02:03:47.762292 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.762240    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55334824-2552-453a-a537-a6a916468bef-metrics-certs\" (UniqueName: \"kubernetes.io/secret/55334824-2552-453a-a537-a6a916468bef-metrics-certs\") pod \"55334824-2552-453a-a537-a6a916468bef\" (UID: \"55334824-2552-453a-a537-a6a916468bef\") "
Nov 04 02:03:47.762500 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.762337    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55334824-2552-453a-a537-a6a916468bef-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55334824-2552-453a-a537-a6a916468bef-maas-api-tls\") pod \"55334824-2552-453a-a537-a6a916468bef\" (UID: \"55334824-2552-453a-a537-a6a916468bef\") "
Nov 04 02:03:47.762500 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.762374    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55334824-2552-453a-a537-a6a916468bef-kube-api-access-t6wg2\" (UniqueName: \"kubernetes.io/projected/55334824-2552-453a-a537-a6a916468bef-kube-api-access-t6wg2\") pod \"55334824-2552-453a-a537-a6a916468bef\" (UID: \"55334824-2552-453a-a537-a6a916468bef\") "
Nov 04 02:03:47.762752 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.762725    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55334824-2552-453a-a537-a6a916468bef-openshift-service-ca" pod "55334824-2552-453a-a537-a6a916468bef" (UID: "55334824-2552-453a-a537-a6a916468bef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:03:47.764488 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.764462    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55334824-2552-453a-a537-a6a916468bef-maas-api-tls" pod "55334824-2552-453a-a537-a6a916468bef" (UID: "55334824-2552-453a-a537-a6a916468bef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:03:47.764488 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.764480    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55334824-2552-453a-a537-a6a916468bef-kube-api-access-t6wg2" pod "55334824-2552-453a-a537-a6a916468bef" (UID: "55334824-2552-453a-a537-a6a916468bef"). InnerVolumeSpecName "kube-api-access-t6wg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:03:47.764612 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.764510    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55334824-2552-453a-a537-a6a916468bef-metrics-certs" pod "55334824-2552-453a-a537-a6a916468bef" (UID: "55334824-2552-453a-a537-a6a916468bef"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:03:47.863684 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.863651    4865 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55334824-2552-453a-a537-a6a916468bef-openshift-service-ca\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:03:47.863684 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.863682    4865 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55334824-2552-453a-a537-a6a916468bef-metrics-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:03:47.863684 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.863693    4865 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55334824-2552-453a-a537-a6a916468bef-maas-api-tls\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:03:47.863906 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.863702    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6wg2\" (UniqueName: \"kubernetes.io/projected/55334824-2552-453a-a537-a6a916468bef-kube-api-access-t6wg2\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:03:47.949795 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.949752    4865 generic.go:356] "Generic (PLEG): container finished" podID="55334824-2552-453a-a537-a6a916468bef" containerID="f36da07cebb2e449e4c0a887b8356ad1262759b6778de9403bcddc5c5acc0a91" exitCode=0
Nov 04 02:03:47.949933 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.949803    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"
Nov 04 02:03:47.949933 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.949819    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh" event={"ID":"55334824-2552-453a-a537-a6a916468bef","Type":"ContainerDied","Data":"f36da07cebb2e449e4c0a887b8356ad1262759b6778de9403bcddc5c5acc0a91"}
Nov 04 02:03:47.949933 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.949866    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh" event={"ID":"55334824-2552-453a-a537-a6a916468bef","Type":"ContainerDied","Data":"bd27bbe0e6ac743fde66e3805d95e4b2f4c200b2071e9ec6a7f48c4d1cf02e95"}
Nov 04 02:03:47.949933 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.949887    4865 scope.go:122] "RemoveContainer" containerID="f36da07cebb2e449e4c0a887b8356ad1262759b6778de9403bcddc5c5acc0a91"
Nov 04 02:03:47.958897 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.958864    4865 scope.go:122] "RemoveContainer" containerID="f36da07cebb2e449e4c0a887b8356ad1262759b6778de9403bcddc5c5acc0a91"
Nov 04 02:03:47.959173 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:03:47.959154    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f36da07cebb2e449e4c0a887b8356ad1262759b6778de9403bcddc5c5acc0a91\": container with ID starting with f36da07cebb2e449e4c0a887b8356ad1262759b6778de9403bcddc5c5acc0a91 not found: ID does not exist" containerID="f36da07cebb2e449e4c0a887b8356ad1262759b6778de9403bcddc5c5acc0a91"
Nov 04 02:03:47.959220 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:47.959183    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f36da07cebb2e449e4c0a887b8356ad1262759b6778de9403bcddc5c5acc0a91"} err="failed to get container status \"f36da07cebb2e449e4c0a887b8356ad1262759b6778de9403bcddc5c5acc0a91\": rpc error: code = NotFound desc = could not find container \"f36da07cebb2e449e4c0a887b8356ad1262759b6778de9403bcddc5c5acc0a91\": container with ID starting with f36da07cebb2e449e4c0a887b8356ad1262759b6778de9403bcddc5c5acc0a91 not found: ID does not exist"
Nov 04 02:03:48.118745 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:48.118663    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"]
Nov 04 02:03:48.122123 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:48.122094    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c69c947c-bf8hh"]
Nov 04 02:03:48.895205 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:03:48.895167    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55334824-2552-453a-a537-a6a916468bef" path="/var/lib/kubelet/pods/55334824-2552-453a-a537-a6a916468bef/volumes"
Nov 04 02:03:58.403740 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:03:58.403699    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:04:28.403656 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:04:28.403615    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:04:58.402510 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:04:58.402469    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:05:28.403320 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:05:28.403262    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:05:36.933990 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:36.933952    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/2.log"
Nov 04 02:05:36.935350 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:36.935329    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/2.log"
Nov 04 02:05:40.161936 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.161903    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"]
Nov 04 02:05:40.162321 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.162220    4865 cpu_manager.go:414] "containerMap: removing container" podUID="d41c1767-9cf0-4a75-b434-b77b6a96d7b3" containerName="authorino"
Nov 04 02:05:40.162321 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.162230    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d41c1767-9cf0-4a75-b434-b77b6a96d7b3" containerName="authorino"
Nov 04 02:05:40.162321 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.162244    4865 cpu_manager.go:414] "containerMap: removing container" podUID="55334824-2552-453a-a537-a6a916468bef" containerName="maas-api"
Nov 04 02:05:40.162321 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.162250    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55334824-2552-453a-a537-a6a916468bef" containerName="maas-api"
Nov 04 02:05:40.162321 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.162263    4865 cpu_manager.go:414] "containerMap: removing container" podUID="26682e91-0320-4953-a6cf-a9abdca87071" containerName="authorino"
Nov 04 02:05:40.162321 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.162267    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26682e91-0320-4953-a6cf-a9abdca87071" containerName="authorino"
Nov 04 02:05:40.162516 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.162346    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="d41c1767-9cf0-4a75-b434-b77b6a96d7b3" containerName="authorino"
Nov 04 02:05:40.162516 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.162355    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="55334824-2552-453a-a537-a6a916468bef" containerName="maas-api"
Nov 04 02:05:40.162516 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.162363    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="26682e91-0320-4953-a6cf-a9abdca87071" containerName="authorino"
Nov 04 02:05:40.165252 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.165235    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"
Nov 04 02:05:40.195490 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.195464    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50e74191-465d-4401-b62c-acce01b9180b-metrics-certs\") pod \"ai-gateway-controller-5d8dddb676-tvx2w\" (UID: \"50e74191-465d-4401-b62c-acce01b9180b\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"
Nov 04 02:05:40.195601 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.195496    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9npf2\" (UniqueName: \"kubernetes.io/projected/50e74191-465d-4401-b62c-acce01b9180b-kube-api-access-9npf2\") pod \"ai-gateway-controller-5d8dddb676-tvx2w\" (UID: \"50e74191-465d-4401-b62c-acce01b9180b\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"
Nov 04 02:05:40.259396 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.259366    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Nov 04 02:05:40.259561 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.259435    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-hvmgq\""
Nov 04 02:05:40.259561 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.259446    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Nov 04 02:05:40.263423 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.263401    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"]
Nov 04 02:05:40.296232 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.296203    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50e74191-465d-4401-b62c-acce01b9180b-metrics-certs\") pod \"ai-gateway-controller-5d8dddb676-tvx2w\" (UID: \"50e74191-465d-4401-b62c-acce01b9180b\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"
Nov 04 02:05:40.296371 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.296240    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9npf2\" (UniqueName: \"kubernetes.io/projected/50e74191-465d-4401-b62c-acce01b9180b-kube-api-access-9npf2\") pod \"ai-gateway-controller-5d8dddb676-tvx2w\" (UID: \"50e74191-465d-4401-b62c-acce01b9180b\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"
Nov 04 02:05:40.298622 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.298603    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50e74191-465d-4401-b62c-acce01b9180b-metrics-certs\") pod \"ai-gateway-controller-5d8dddb676-tvx2w\" (UID: \"50e74191-465d-4401-b62c-acce01b9180b\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"
Nov 04 02:05:40.358210 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.358179    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9npf2\" (UniqueName: \"kubernetes.io/projected/50e74191-465d-4401-b62c-acce01b9180b-kube-api-access-9npf2\") pod \"ai-gateway-controller-5d8dddb676-tvx2w\" (UID: \"50e74191-465d-4401-b62c-acce01b9180b\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"
Nov 04 02:05:40.477157 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.477130    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"
Nov 04 02:05:40.851313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:40.848100    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"]
Nov 04 02:05:41.427643 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:41.427608    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w" event={"ID":"50e74191-465d-4401-b62c-acce01b9180b","Type":"ContainerStarted","Data":"39d0b428547f59096262343b515cc3581cbe7f258500edde8bfe64aadf4d2608"}
Nov 04 02:05:43.437658 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:43.437626    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w" event={"ID":"50e74191-465d-4401-b62c-acce01b9180b","Type":"ContainerStarted","Data":"b1d728c82b18630e31b7101e0a759fcad583b90ee82c14797d7df00247da3a10"}
Nov 04 02:05:43.438057 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:43.437758    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"
Nov 04 02:05:43.556560 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:43.556463    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w" podStartSLOduration=1.213542272 podStartE2EDuration="3.556449172s" totalImagesPullingTime="2.3429069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:05:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:43.555243496 +0000 UTC m=+606.782030165" watchObservedRunningTime="2026-11-04 02:05:43.556449172 +0000 UTC m=+606.783235834"
Nov 04 02:05:47.536277 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.536242    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-7hkvk"]
Nov 04 02:05:47.539455 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.539438    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7hkvk"
Nov 04 02:05:47.592870 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.592838    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Nov 04 02:05:47.643717 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.643688    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-qsgqw\""
Nov 04 02:05:47.644923 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.644902    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-7hkvk"]
Nov 04 02:05:47.660617 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.660589    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0032af18-ab69-46ad-b078-d61f927b22ca-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-7hkvk\" (UID: \"0032af18-ab69-46ad-b078-d61f927b22ca\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7hkvk"
Nov 04 02:05:47.660716 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.660640    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqpmf\" (UniqueName: \"kubernetes.io/projected/0032af18-ab69-46ad-b078-d61f927b22ca-kube-api-access-wqpmf\") pod \"payload-pre-processing-7b98bb678c-7hkvk\" (UID: \"0032af18-ab69-46ad-b078-d61f927b22ca\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7hkvk"
Nov 04 02:05:47.696252 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.696225    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-ckdnq"]
Nov 04 02:05:47.701538 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.701509    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-ckdnq"
Nov 04 02:05:47.739955 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.739924    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-ckdnq"]
Nov 04 02:05:47.761225 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.761196    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqpmf\" (UniqueName: \"kubernetes.io/projected/0032af18-ab69-46ad-b078-d61f927b22ca-kube-api-access-wqpmf\") pod \"payload-pre-processing-7b98bb678c-7hkvk\" (UID: \"0032af18-ab69-46ad-b078-d61f927b22ca\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7hkvk"
Nov 04 02:05:47.761374 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.761274    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70290b6d-1703-43ad-a7e5-42c79eed7e1a-plugins-config-volume\") pod \"payload-processing-7bb9489676-ckdnq\" (UID: \"70290b6d-1703-43ad-a7e5-42c79eed7e1a\") " pod="openshift-ingress/payload-processing-7bb9489676-ckdnq"
Nov 04 02:05:47.761445 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.761370    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfddg\" (UniqueName: \"kubernetes.io/projected/70290b6d-1703-43ad-a7e5-42c79eed7e1a-kube-api-access-nfddg\") pod \"payload-processing-7bb9489676-ckdnq\" (UID: \"70290b6d-1703-43ad-a7e5-42c79eed7e1a\") " pod="openshift-ingress/payload-processing-7bb9489676-ckdnq"
Nov 04 02:05:47.761445 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.761419    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0032af18-ab69-46ad-b078-d61f927b22ca-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-7hkvk\" (UID: \"0032af18-ab69-46ad-b078-d61f927b22ca\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7hkvk"
Nov 04 02:05:47.762047 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.762030    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0032af18-ab69-46ad-b078-d61f927b22ca-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-7hkvk\" (UID: \"0032af18-ab69-46ad-b078-d61f927b22ca\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7hkvk"
Nov 04 02:05:47.825927 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.825876    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqpmf\" (UniqueName: \"kubernetes.io/projected/0032af18-ab69-46ad-b078-d61f927b22ca-kube-api-access-wqpmf\") pod \"payload-pre-processing-7b98bb678c-7hkvk\" (UID: \"0032af18-ab69-46ad-b078-d61f927b22ca\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7hkvk"
Nov 04 02:05:47.851721 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.851702    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7hkvk"
Nov 04 02:05:47.862113 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.861960    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70290b6d-1703-43ad-a7e5-42c79eed7e1a-plugins-config-volume\") pod \"payload-processing-7bb9489676-ckdnq\" (UID: \"70290b6d-1703-43ad-a7e5-42c79eed7e1a\") " pod="openshift-ingress/payload-processing-7bb9489676-ckdnq"
Nov 04 02:05:47.862113 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.862004    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfddg\" (UniqueName: \"kubernetes.io/projected/70290b6d-1703-43ad-a7e5-42c79eed7e1a-kube-api-access-nfddg\") pod \"payload-processing-7bb9489676-ckdnq\" (UID: \"70290b6d-1703-43ad-a7e5-42c79eed7e1a\") " pod="openshift-ingress/payload-processing-7bb9489676-ckdnq"
Nov 04 02:05:47.862713 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.862691    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70290b6d-1703-43ad-a7e5-42c79eed7e1a-plugins-config-volume\") pod \"payload-processing-7bb9489676-ckdnq\" (UID: \"70290b6d-1703-43ad-a7e5-42c79eed7e1a\") " pod="openshift-ingress/payload-processing-7bb9489676-ckdnq"
Nov 04 02:05:47.925250 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:47.925222    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfddg\" (UniqueName: \"kubernetes.io/projected/70290b6d-1703-43ad-a7e5-42c79eed7e1a-kube-api-access-nfddg\") pod \"payload-processing-7bb9489676-ckdnq\" (UID: \"70290b6d-1703-43ad-a7e5-42c79eed7e1a\") " pod="openshift-ingress/payload-processing-7bb9489676-ckdnq"
Nov 04 02:05:48.015238 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:48.015206    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-ckdnq"
Nov 04 02:05:48.222391 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:48.222365    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-7hkvk"]
Nov 04 02:05:48.224892 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:05:48.224866    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0032af18_ab69_46ad_b078_d61f927b22ca.slice/crio-014e31a46c937564238bfd49be4aa6734d0e1f16c8f82507aff9c687f8ac2307 WatchSource:0}: Error finding container 014e31a46c937564238bfd49be4aa6734d0e1f16c8f82507aff9c687f8ac2307: Status 404 returned error can't find the container with id 014e31a46c937564238bfd49be4aa6734d0e1f16c8f82507aff9c687f8ac2307
Nov 04 02:05:48.226700 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:48.226681    4865 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:05:48.383879 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:48.383789    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-ckdnq"]
Nov 04 02:05:48.386406 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:05:48.386380    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70290b6d_1703_43ad_a7e5_42c79eed7e1a.slice/crio-8a18722bd63040642dc6a0673be67d2e90c338e34ddc12b3880e954f055c662c WatchSource:0}: Error finding container 8a18722bd63040642dc6a0673be67d2e90c338e34ddc12b3880e954f055c662c: Status 404 returned error can't find the container with id 8a18722bd63040642dc6a0673be67d2e90c338e34ddc12b3880e954f055c662c
Nov 04 02:05:48.458846 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:48.458796    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-ckdnq" event={"ID":"70290b6d-1703-43ad-a7e5-42c79eed7e1a","Type":"ContainerStarted","Data":"8a18722bd63040642dc6a0673be67d2e90c338e34ddc12b3880e954f055c662c"}
Nov 04 02:05:48.459893 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:48.459869    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7hkvk" event={"ID":"0032af18-ab69-46ad-b078-d61f927b22ca","Type":"ContainerStarted","Data":"014e31a46c937564238bfd49be4aa6734d0e1f16c8f82507aff9c687f8ac2307"}
Nov 04 02:05:51.477080 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:51.477051    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-ckdnq" event={"ID":"70290b6d-1703-43ad-a7e5-42c79eed7e1a","Type":"ContainerStarted","Data":"79205003a8e63891b3ab342230c77a5d92086abfbef0273cf5eebbb35ba82c98"}
Nov 04 02:05:51.477415 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:51.477130    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bb9489676-ckdnq"
Nov 04 02:05:51.595230 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:51.595182    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7bb9489676-ckdnq" podStartSLOduration=1.626776997 podStartE2EDuration="4.595169885s" totalImagesPullingTime="2.968392888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:05:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:51.594242521 +0000 UTC m=+614.821029188" watchObservedRunningTime="2026-11-04 02:05:51.595169885 +0000 UTC m=+614.821956553"
Nov 04 02:05:52.482741 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:52.482707    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7hkvk" event={"ID":"0032af18-ab69-46ad-b078-d61f927b22ca","Type":"ContainerStarted","Data":"18647aa7b44fc89f96a516f1e321500a603b1c5bf7367614e18730702736fba0"}
Nov 04 02:05:52.483137 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:52.482802    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7hkvk"
Nov 04 02:05:52.601829 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:52.601769    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7hkvk" podStartSLOduration=2.409691507 podStartE2EDuration="5.60175631s" totalImagesPullingTime="3.192064803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:05:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:52.600915279 +0000 UTC m=+615.827701948" watchObservedRunningTime="2026-11-04 02:05:52.60175631 +0000 UTC m=+615.828542978"
Nov 04 02:05:54.444097 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:54.444065    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"
Nov 04 02:05:54.833831 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:54.833742    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7656484495-trb67"]
Nov 04 02:05:54.837265 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:54.837245    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7656484495-trb67"
Nov 04 02:05:54.895835 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:54.895810    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7656484495-trb67"]
Nov 04 02:05:54.928458 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:54.928432    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/574b6242-13d5-467e-b940-e67d73036f08-metrics-certs\") pod \"ai-gateway-controller-7656484495-trb67\" (UID: \"574b6242-13d5-467e-b940-e67d73036f08\") " pod="opendatahub/ai-gateway-controller-7656484495-trb67"
Nov 04 02:05:54.928594 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:54.928495    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j5bj\" (UniqueName: \"kubernetes.io/projected/574b6242-13d5-467e-b940-e67d73036f08-kube-api-access-4j5bj\") pod \"ai-gateway-controller-7656484495-trb67\" (UID: \"574b6242-13d5-467e-b940-e67d73036f08\") " pod="opendatahub/ai-gateway-controller-7656484495-trb67"
Nov 04 02:05:55.029513 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:55.029480    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j5bj\" (UniqueName: \"kubernetes.io/projected/574b6242-13d5-467e-b940-e67d73036f08-kube-api-access-4j5bj\") pod \"ai-gateway-controller-7656484495-trb67\" (UID: \"574b6242-13d5-467e-b940-e67d73036f08\") " pod="opendatahub/ai-gateway-controller-7656484495-trb67"
Nov 04 02:05:55.029676 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:55.029532    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/574b6242-13d5-467e-b940-e67d73036f08-metrics-certs\") pod \"ai-gateway-controller-7656484495-trb67\" (UID: \"574b6242-13d5-467e-b940-e67d73036f08\") " pod="opendatahub/ai-gateway-controller-7656484495-trb67"
Nov 04 02:05:55.031960 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:55.031933    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/574b6242-13d5-467e-b940-e67d73036f08-metrics-certs\") pod \"ai-gateway-controller-7656484495-trb67\" (UID: \"574b6242-13d5-467e-b940-e67d73036f08\") " pod="opendatahub/ai-gateway-controller-7656484495-trb67"
Nov 04 02:05:55.092662 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:55.092601    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j5bj\" (UniqueName: \"kubernetes.io/projected/574b6242-13d5-467e-b940-e67d73036f08-kube-api-access-4j5bj\") pod \"ai-gateway-controller-7656484495-trb67\" (UID: \"574b6242-13d5-467e-b940-e67d73036f08\") " pod="opendatahub/ai-gateway-controller-7656484495-trb67"
Nov 04 02:05:55.149572 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:55.149528    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7656484495-trb67"
Nov 04 02:05:55.519660 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:55.519630    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7656484495-trb67"]
Nov 04 02:05:55.522486 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:05:55.522460    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod574b6242_13d5_467e_b940_e67d73036f08.slice/crio-2084eab8b50aa46b14953fe055c556ec546e94bff5fe5deb21f3c3c559aca1a0 WatchSource:0}: Error finding container 2084eab8b50aa46b14953fe055c556ec546e94bff5fe5deb21f3c3c559aca1a0: Status 404 returned error can't find the container with id 2084eab8b50aa46b14953fe055c556ec546e94bff5fe5deb21f3c3c559aca1a0
Nov 04 02:05:56.500903 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:56.500862    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7656484495-trb67" event={"ID":"574b6242-13d5-467e-b940-e67d73036f08","Type":"ContainerStarted","Data":"464c2e529428d6a35f6c88626c46b075f109b3471ddb043fd31c7489eb131630"}
Nov 04 02:05:56.501082 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:56.500918    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7656484495-trb67" event={"ID":"574b6242-13d5-467e-b940-e67d73036f08","Type":"ContainerStarted","Data":"2084eab8b50aa46b14953fe055c556ec546e94bff5fe5deb21f3c3c559aca1a0"}
Nov 04 02:05:56.501082 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:56.501003    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7656484495-trb67"
Nov 04 02:05:56.619027 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:05:56.618913    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7656484495-trb67" podStartSLOduration=1.859042433 podStartE2EDuration="2.618897742s" totalImagesPullingTime="759.855309ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:05:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:56.618240149 +0000 UTC m=+619.845026818" watchObservedRunningTime="2026-11-04 02:05:56.618897742 +0000 UTC m=+619.845684409"
Nov 04 02:05:58.403808 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:05:58.403760    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:06:02.486687 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:02.486651    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bb9489676-ckdnq"
Nov 04 02:06:03.489887 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:03.489860    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7hkvk"
Nov 04 02:06:07.507179 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:07.507148    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7656484495-trb67"
Nov 04 02:06:07.677540 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:07.677506    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"]
Nov 04 02:06:07.677779 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:07.677734    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w" podUID="50e74191-465d-4401-b62c-acce01b9180b" containerName="manager" containerID="cri-o://b1d728c82b18630e31b7101e0a759fcad583b90ee82c14797d7df00247da3a10" gracePeriod=30
Nov 04 02:06:07.919619 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:07.919597    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"
Nov 04 02:06:08.033162 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:08.033127    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/50e74191-465d-4401-b62c-acce01b9180b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/50e74191-465d-4401-b62c-acce01b9180b-metrics-certs\") pod \"50e74191-465d-4401-b62c-acce01b9180b\" (UID: \"50e74191-465d-4401-b62c-acce01b9180b\") "
Nov 04 02:06:08.033351 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:08.033199    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50e74191-465d-4401-b62c-acce01b9180b-kube-api-access-9npf2\" (UniqueName: \"kubernetes.io/projected/50e74191-465d-4401-b62c-acce01b9180b-kube-api-access-9npf2\") pod \"50e74191-465d-4401-b62c-acce01b9180b\" (UID: \"50e74191-465d-4401-b62c-acce01b9180b\") "
Nov 04 02:06:08.035430 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:08.035405    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50e74191-465d-4401-b62c-acce01b9180b-metrics-certs" pod "50e74191-465d-4401-b62c-acce01b9180b" (UID: "50e74191-465d-4401-b62c-acce01b9180b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:06:08.035561 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:08.035499    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50e74191-465d-4401-b62c-acce01b9180b-kube-api-access-9npf2" pod "50e74191-465d-4401-b62c-acce01b9180b" (UID: "50e74191-465d-4401-b62c-acce01b9180b"). InnerVolumeSpecName "kube-api-access-9npf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:06:08.134131 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:08.134055    4865 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50e74191-465d-4401-b62c-acce01b9180b-metrics-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:06:08.134131 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:08.134085    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9npf2\" (UniqueName: \"kubernetes.io/projected/50e74191-465d-4401-b62c-acce01b9180b-kube-api-access-9npf2\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:06:08.551171 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:08.551134    4865 generic.go:356] "Generic (PLEG): container finished" podID="50e74191-465d-4401-b62c-acce01b9180b" containerID="b1d728c82b18630e31b7101e0a759fcad583b90ee82c14797d7df00247da3a10" exitCode=0
Nov 04 02:06:08.551598 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:08.551180    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"
Nov 04 02:06:08.551598 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:08.551223    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w" event={"ID":"50e74191-465d-4401-b62c-acce01b9180b","Type":"ContainerDied","Data":"b1d728c82b18630e31b7101e0a759fcad583b90ee82c14797d7df00247da3a10"}
Nov 04 02:06:08.551598 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:08.551274    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d8dddb676-tvx2w" event={"ID":"50e74191-465d-4401-b62c-acce01b9180b","Type":"ContainerDied","Data":"39d0b428547f59096262343b515cc3581cbe7f258500edde8bfe64aadf4d2608"}
Nov 04 02:06:08.551598 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:08.551316    4865 scope.go:122] "RemoveContainer" containerID="b1d728c82b18630e31b7101e0a759fcad583b90ee82c14797d7df00247da3a10"
Nov 04 02:06:08.560675 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:08.560654    4865 scope.go:122] "RemoveContainer" containerID="b1d728c82b18630e31b7101e0a759fcad583b90ee82c14797d7df00247da3a10"
Nov 04 02:06:08.560913 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:06:08.560896    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1d728c82b18630e31b7101e0a759fcad583b90ee82c14797d7df00247da3a10\": container with ID starting with b1d728c82b18630e31b7101e0a759fcad583b90ee82c14797d7df00247da3a10 not found: ID does not exist" containerID="b1d728c82b18630e31b7101e0a759fcad583b90ee82c14797d7df00247da3a10"
Nov 04 02:06:08.560966 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:08.560921    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1d728c82b18630e31b7101e0a759fcad583b90ee82c14797d7df00247da3a10"} err="failed to get container status \"b1d728c82b18630e31b7101e0a759fcad583b90ee82c14797d7df00247da3a10\": rpc error: code = NotFound desc = could not find container \"b1d728c82b18630e31b7101e0a759fcad583b90ee82c14797d7df00247da3a10\": container with ID starting with b1d728c82b18630e31b7101e0a759fcad583b90ee82c14797d7df00247da3a10 not found: ID does not exist"
Nov 04 02:06:08.718361 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:08.718332    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"]
Nov 04 02:06:08.769132 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:08.769105    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-tvx2w"]
Nov 04 02:06:08.894080 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:08.894011    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="50e74191-465d-4401-b62c-acce01b9180b" path="/var/lib/kubelet/pods/50e74191-465d-4401-b62c-acce01b9180b/volumes"
Nov 04 02:06:28.402450 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:06:28.402414    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:06:45.226132 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:45.226056    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:06:49.406425 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.406393    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"]
Nov 04 02:06:49.406808 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.406727    4865 cpu_manager.go:414] "containerMap: removing container" podUID="50e74191-465d-4401-b62c-acce01b9180b" containerName="manager"
Nov 04 02:06:49.406808 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.406737    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50e74191-465d-4401-b62c-acce01b9180b" containerName="manager"
Nov 04 02:06:49.406885 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.406826    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="50e74191-465d-4401-b62c-acce01b9180b" containerName="manager"
Nov 04 02:06:49.409586 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.409569    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.471332 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.471273    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54832357-9824-4b08-b0a6-f58dcc2f768b-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.471503 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.471365    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkxcv\" (UniqueName: \"kubernetes.io/projected/54832357-9824-4b08-b0a6-f58dcc2f768b-kube-api-access-hkxcv\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.471548 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.471492    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54832357-9824-4b08-b0a6-f58dcc2f768b-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.471548 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.471535    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54832357-9824-4b08-b0a6-f58dcc2f768b-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.471629 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.471614    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54832357-9824-4b08-b0a6-f58dcc2f768b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.471738 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.471637    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54832357-9824-4b08-b0a6-f58dcc2f768b-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.513578 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.513530    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 04 02:06:49.513578 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.513564    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 04 02:06:49.513767 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.513570    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Nov 04 02:06:49.513767 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.513687    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rczvg\""
Nov 04 02:06:49.515136 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.515116    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"]
Nov 04 02:06:49.572183 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.572154    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54832357-9824-4b08-b0a6-f58dcc2f768b-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.572359 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.572185    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkxcv\" (UniqueName: \"kubernetes.io/projected/54832357-9824-4b08-b0a6-f58dcc2f768b-kube-api-access-hkxcv\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.572359 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.572234    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54832357-9824-4b08-b0a6-f58dcc2f768b-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.572359 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.572262    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54832357-9824-4b08-b0a6-f58dcc2f768b-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.572359 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.572312    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54832357-9824-4b08-b0a6-f58dcc2f768b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.572359 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.572337    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54832357-9824-4b08-b0a6-f58dcc2f768b-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.572689 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.572663    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54832357-9824-4b08-b0a6-f58dcc2f768b-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.572735 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.572715    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54832357-9824-4b08-b0a6-f58dcc2f768b-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.572796 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.572742    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54832357-9824-4b08-b0a6-f58dcc2f768b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.574438 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.574414    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54832357-9824-4b08-b0a6-f58dcc2f768b-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.574694 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.574675    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54832357-9824-4b08-b0a6-f58dcc2f768b-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.633993 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.633963    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkxcv\" (UniqueName: \"kubernetes.io/projected/54832357-9824-4b08-b0a6-f58dcc2f768b-kube-api-access-hkxcv\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-c7flb\" (UID: \"54832357-9824-4b08-b0a6-f58dcc2f768b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:49.722152 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:49.722116    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:50.093703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:50.093675    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"]
Nov 04 02:06:50.095764 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:06:50.095732    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54832357_9824_4b08_b0a6_f58dcc2f768b.slice/crio-319cf990220985dadcb209182c314f2bac13dffd067a6d6f7446e27d845220ab WatchSource:0}: Error finding container 319cf990220985dadcb209182c314f2bac13dffd067a6d6f7446e27d845220ab: Status 404 returned error can't find the container with id 319cf990220985dadcb209182c314f2bac13dffd067a6d6f7446e27d845220ab
Nov 04 02:06:50.743461 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:50.743421    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb" event={"ID":"54832357-9824-4b08-b0a6-f58dcc2f768b","Type":"ContainerStarted","Data":"319cf990220985dadcb209182c314f2bac13dffd067a6d6f7446e27d845220ab"}
Nov 04 02:06:51.750133 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:51.750093    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb" event={"ID":"54832357-9824-4b08-b0a6-f58dcc2f768b","Type":"ContainerStarted","Data":"10eaf4b135cc763f071c22fe32c57877bff6e177c8047c27cbccbe9193bf7c6f"}
Nov 04 02:06:51.869623 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:51.869532    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb" podStartSLOduration=1.380885506 podStartE2EDuration="2.869516551s" totalImagesPullingTime="1.488631045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:51.868869239 +0000 UTC m=+675.095655907" watchObservedRunningTime="2026-11-04 02:06:51.869516551 +0000 UTC m=+675.096303219"
Nov 04 02:06:52.722631 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:52.722596    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:52.828900 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:52.828859    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:06:58.403366 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:06:58.403327    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:06:58.963672 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:58.963639    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"]
Nov 04 02:06:58.967321 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:58.967299    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.026392 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.026351    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Nov 04 02:06:59.067108 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.067067    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.067377 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.067133    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.067377 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.067222    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.067377 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.067257    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.067377 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.067330    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.067377 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.067357    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2sl5\" (UniqueName: \"kubernetes.io/projected/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-kube-api-access-n2sl5\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.075243 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.075209    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"]
Nov 04 02:06:59.167823 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.167766    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.168056 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.167853    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.168056 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.167884    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.168056 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.167923    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2sl5\" (UniqueName: \"kubernetes.io/projected/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-kube-api-access-n2sl5\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.168056 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.167961    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.168056 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.168020    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.168841 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.168812    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.169043 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.168919    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.169346 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.169065    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.171020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.170995    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.171216 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.171200    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.233482 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.233407    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2sl5\" (UniqueName: \"kubernetes.io/projected/2c1f5846-cfc2-4b97-b24e-c2b99105ef98-kube-api-access-n2sl5\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9cbzm\" (UID: \"2c1f5846-cfc2-4b97-b24e-c2b99105ef98\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.281273 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.281234    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:06:59.664686 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.664656    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"]
Nov 04 02:06:59.667117 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:06:59.667087    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c1f5846_cfc2_4b97_b24e_c2b99105ef98.slice/crio-3490ebbb011d0a77420e783d5eb11c0b32833bb75a7e829336d4151a6a385f51 WatchSource:0}: Error finding container 3490ebbb011d0a77420e783d5eb11c0b32833bb75a7e829336d4151a6a385f51: Status 404 returned error can't find the container with id 3490ebbb011d0a77420e783d5eb11c0b32833bb75a7e829336d4151a6a385f51
Nov 04 02:06:59.722398 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.722367    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:59.735491 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.735461    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:06:59.789419 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.789380    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm" event={"ID":"2c1f5846-cfc2-4b97-b24e-c2b99105ef98","Type":"ContainerStarted","Data":"3490ebbb011d0a77420e783d5eb11c0b32833bb75a7e829336d4151a6a385f51"}
Nov 04 02:06:59.800918 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:06:59.800896    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb"
Nov 04 02:07:00.796992 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:00.796937    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm" event={"ID":"2c1f5846-cfc2-4b97-b24e-c2b99105ef98","Type":"ContainerStarted","Data":"7114a47b19e2c876c883145e496209b98ef8e754f2f75b19999ae6f16c1980a6"}
Nov 04 02:07:00.916955 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:00.916905    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm" podStartSLOduration=2.320912777 podStartE2EDuration="2.916891322s" totalImagesPullingTime="595.978545ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:07:00.916512312 +0000 UTC m=+684.143298980" watchObservedRunningTime="2026-11-04 02:07:00.916891322 +0000 UTC m=+684.143677990"
Nov 04 02:07:01.281926 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:01.281889    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:07:02.005689 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.005653    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"]
Nov 04 02:07:02.009126 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.009105    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.067742 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.067711    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Nov 04 02:07:02.108792 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.108753    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.108931 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.108808    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.108931 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.108904    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.109014 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.108958    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfw9s\" (UniqueName: \"kubernetes.io/projected/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-kube-api-access-sfw9s\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.109049 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.109016    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.109087 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.109073    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.119523 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.119496    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"]
Nov 04 02:07:02.210258 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.210221    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.210258 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.210258    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.210531 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.210317    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfw9s\" (UniqueName: \"kubernetes.io/projected/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-kube-api-access-sfw9s\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.210531 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.210347    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.210531 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.210369    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.210531 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.210432    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.210737 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.210680    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.210737 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.210703    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.210804 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.210781    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.212668 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.212648    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.212970 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.212949    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.272381 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.272311    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfw9s\" (UniqueName: \"kubernetes.io/projected/15039fbe-55b4-4b82-9b2d-78d24b0ed5b3-kube-api-access-sfw9s\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl\" (UID: \"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.321464 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.321421    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:02.693341 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.693311    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"]
Nov 04 02:07:02.695592 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:07:02.695565    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15039fbe_55b4_4b82_9b2d_78d24b0ed5b3.slice/crio-502c520df6578aa384aebb50249fc6d342864a4dd62c9c2844bcc6a382dd4eca WatchSource:0}: Error finding container 502c520df6578aa384aebb50249fc6d342864a4dd62c9c2844bcc6a382dd4eca: Status 404 returned error can't find the container with id 502c520df6578aa384aebb50249fc6d342864a4dd62c9c2844bcc6a382dd4eca
Nov 04 02:07:02.813436 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:02.813389    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl" event={"ID":"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3","Type":"ContainerStarted","Data":"502c520df6578aa384aebb50249fc6d342864a4dd62c9c2844bcc6a382dd4eca"}
Nov 04 02:07:03.820030 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:03.819992    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl" event={"ID":"15039fbe-55b4-4b82-9b2d-78d24b0ed5b3","Type":"ContainerStarted","Data":"5cf88cad72c81977166df0eb56f93a1e13285e804c3af7df0448fe2c66a5b2f2"}
Nov 04 02:07:03.940535 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:03.940480    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl" podStartSLOduration=2.354707732 podStartE2EDuration="2.940468079s" totalImagesPullingTime="585.760347ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:07:03.939725599 +0000 UTC m=+687.166512267" watchObservedRunningTime="2026-11-04 02:07:03.940468079 +0000 UTC m=+687.167254747"
Nov 04 02:07:04.322508 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:04.322463    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:04.429724 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:04.429684    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:07:07.332041 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.332006    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"]
Nov 04 02:07:07.335661 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.335644    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.389791 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.389749    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wgllv\""
Nov 04 02:07:07.441858 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.441823    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"]
Nov 04 02:07:07.461448 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.461417    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.461618 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.461482    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.461618 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.461564    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzvr8\" (UniqueName: \"kubernetes.io/projected/52289639-1d6e-4d31-b2da-49277feea25d-kube-api-access-jzvr8\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.461706 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.461616    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52289639-1d6e-4d31-b2da-49277feea25d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.461745 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.461708    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.461792 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.461759    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52289639-1d6e-4d31-b2da-49277feea25d-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.461839 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.461795    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.461839 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.461814    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52289639-1d6e-4d31-b2da-49277feea25d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.461923 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.461895    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.562966 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.562927    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52289639-1d6e-4d31-b2da-49277feea25d-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.562966 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.562969    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.563218 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.562994    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52289639-1d6e-4d31-b2da-49277feea25d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.563218 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.563043    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.563218 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.563064    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.563218 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.563125    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.563218 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.563159    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzvr8\" (UniqueName: \"kubernetes.io/projected/52289639-1d6e-4d31-b2da-49277feea25d-kube-api-access-jzvr8\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.563218 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.563214    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52289639-1d6e-4d31-b2da-49277feea25d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.563610 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.563240    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.563610 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.563532    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.563715 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.563691    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.563765 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.563724    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.563801 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.563772    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.563922 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.563902    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52289639-1d6e-4d31-b2da-49277feea25d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.565324 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.565301    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.565627 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.565608    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52289639-1d6e-4d31-b2da-49277feea25d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.624881 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.624807    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52289639-1d6e-4d31-b2da-49277feea25d-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.624881 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.624873    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzvr8\" (UniqueName: \"kubernetes.io/projected/52289639-1d6e-4d31-b2da-49277feea25d-kube-api-access-jzvr8\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8h87v\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:07.648783 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:07.648745    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:08.074650 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:08.074367    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"]
Nov 04 02:07:08.076894 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:07:08.076865    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52289639_1d6e_4d31_b2da_49277feea25d.slice/crio-595757bffb4fa69f6b056b35688a5f52ca6f98728fc88c9ab2396124361b47ca WatchSource:0}: Error finding container 595757bffb4fa69f6b056b35688a5f52ca6f98728fc88c9ab2396124361b47ca: Status 404 returned error can't find the container with id 595757bffb4fa69f6b056b35688a5f52ca6f98728fc88c9ab2396124361b47ca
Nov 04 02:07:08.844675 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:08.844636    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v" event={"ID":"52289639-1d6e-4d31-b2da-49277feea25d","Type":"ContainerStarted","Data":"595757bffb4fa69f6b056b35688a5f52ca6f98728fc88c9ab2396124361b47ca"}
Nov 04 02:07:09.282078 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:09.282042    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:07:09.294815 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:09.294785    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:07:09.860266 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:09.860238    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm"
Nov 04 02:07:12.143687 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:12.143655    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:07:12.143989 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:12.143723    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:07:12.143989 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:12.143753    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:07:12.322641 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:12.322591    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:12.335784 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:12.335755    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:12.867390 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:12.867349    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v" event={"ID":"52289639-1d6e-4d31-b2da-49277feea25d","Type":"ContainerStarted","Data":"277219f201d104d648af53662f0c6662c39e13db499d6ff180003b9b4d0c657d"}
Nov 04 02:07:12.878566 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:12.878538    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl"
Nov 04 02:07:12.992154 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:12.992099    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v" podStartSLOduration=1.928158606 podStartE2EDuration="5.992084757s" totalImagesPullingTime="4.063926151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:07:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:07:12.989563746 +0000 UTC m=+696.216350416" watchObservedRunningTime="2026-11-04 02:07:12.992084757 +0000 UTC m=+696.218871424"
Nov 04 02:07:13.656631 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:13.656598    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:13.657926 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:13.657901    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Nov 04 02:07:13.658032 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:13.657947    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v" podUID="52289639-1d6e-4d31-b2da-49277feea25d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Nov 04 02:07:14.652422 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:14.652383    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Nov 04 02:07:14.656408 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:14.652678    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v" podUID="52289639-1d6e-4d31-b2da-49277feea25d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Nov 04 02:07:15.158701 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.158661    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"]
Nov 04 02:07:15.162927 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.162894    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.218859 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.218826    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"]
Nov 04 02:07:15.227930 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.227900    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqwjx\" (UniqueName: \"kubernetes.io/projected/f7dc3968-3684-408e-9d43-568adb008d30-kube-api-access-xqwjx\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.228086 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.227948    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7dc3968-3684-408e-9d43-568adb008d30-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.228086 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.227976    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7dc3968-3684-408e-9d43-568adb008d30-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.228086 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.228003    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7dc3968-3684-408e-9d43-568adb008d30-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.228086 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.228037    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7dc3968-3684-408e-9d43-568adb008d30-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.228086 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.228071    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7dc3968-3684-408e-9d43-568adb008d30-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.267453 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.267419    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Nov 04 02:07:15.329127 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.329087    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7dc3968-3684-408e-9d43-568adb008d30-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.329347 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.329149    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7dc3968-3684-408e-9d43-568adb008d30-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.329347 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.329185    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7dc3968-3684-408e-9d43-568adb008d30-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.329347 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.329230    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7dc3968-3684-408e-9d43-568adb008d30-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.329347 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.329258    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqwjx\" (UniqueName: \"kubernetes.io/projected/f7dc3968-3684-408e-9d43-568adb008d30-kube-api-access-xqwjx\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.329347 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.329320    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7dc3968-3684-408e-9d43-568adb008d30-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.329778 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.329735    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7dc3968-3684-408e-9d43-568adb008d30-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.329903 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.329797    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7dc3968-3684-408e-9d43-568adb008d30-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.329903 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.329883    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7dc3968-3684-408e-9d43-568adb008d30-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.340301 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.331546    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7dc3968-3684-408e-9d43-568adb008d30-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.340301 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.338899    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7dc3968-3684-408e-9d43-568adb008d30-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.391803 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.391772    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqwjx\" (UniqueName: \"kubernetes.io/projected/f7dc3968-3684-408e-9d43-568adb008d30-kube-api-access-xqwjx\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp\" (UID: \"f7dc3968-3684-408e-9d43-568adb008d30\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.484305 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.484221    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:15.652311 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.649679    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Nov 04 02:07:15.652311 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.649740    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v" podUID="52289639-1d6e-4d31-b2da-49277feea25d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Nov 04 02:07:15.953361 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:15.952237    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"]
Nov 04 02:07:16.654305 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:16.652699    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Nov 04 02:07:16.654305 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:16.652762    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v" podUID="52289639-1d6e-4d31-b2da-49277feea25d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Nov 04 02:07:16.895363 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:16.895265    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp" event={"ID":"f7dc3968-3684-408e-9d43-568adb008d30","Type":"ContainerStarted","Data":"666581bcffc750946d8d3a501abfe4a1abf13bcd82954e66ca4498e176d6adee"}
Nov 04 02:07:16.895363 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:16.895323    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp" event={"ID":"f7dc3968-3684-408e-9d43-568adb008d30","Type":"ContainerStarted","Data":"fa7cf182a36f12d78c322023945db48ca5648fae2cf9fcfa331d4cfb2f39ce38"}
Nov 04 02:07:17.019151 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:17.018624    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp" podStartSLOduration=1.378269765 podStartE2EDuration="2.018608136s" totalImagesPullingTime="640.338371ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:07:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:07:17.017417278 +0000 UTC m=+700.244203949" watchObservedRunningTime="2026-11-04 02:07:17.018608136 +0000 UTC m=+700.245394803"
Nov 04 02:07:17.485108 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:17.485076    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:17.652195 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:17.651750    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Nov 04 02:07:17.652195 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:17.651809    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v" podUID="52289639-1d6e-4d31-b2da-49277feea25d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Nov 04 02:07:18.649487 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:18.649453    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Nov 04 02:07:18.649896 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:18.649504    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v" podUID="52289639-1d6e-4d31-b2da-49277feea25d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Nov 04 02:07:20.650383 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:20.650334    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:07:20.650767 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:20.650416    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v" podUID="52289639-1d6e-4d31-b2da-49277feea25d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:07:21.693447 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:21.693402    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:07:21.693924 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:21.693473    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v" podUID="52289639-1d6e-4d31-b2da-49277feea25d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:07:21.949124 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:21.949035    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:21.949507 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:21.949472    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:21.950812 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:21.950767    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:07:22.057127 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:22.057096    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:07:25.484766 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:25.484731    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:25.496819 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:25.496795    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:25.943300 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:25.943260    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp"
Nov 04 02:07:28.402954 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:07:28.402920    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:07:37.333113 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.333079    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"]
Nov 04 02:07:37.338460 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.338443    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.445586 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.445549    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.445746 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.445603    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.445746 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.445681    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.445829 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.445809    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.445863 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.445833    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/627ad113-e0fa-4637-9ac6-c71f11f43974-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.445863 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.445852    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snts4\" (UniqueName: \"kubernetes.io/projected/627ad113-e0fa-4637-9ac6-c71f11f43974-kube-api-access-snts4\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.445947 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.445873    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.445947 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.445905    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.446052 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.446001    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.547209 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.547174    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.547425 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.547276    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/627ad113-e0fa-4637-9ac6-c71f11f43974-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.547425 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.547329    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snts4\" (UniqueName: \"kubernetes.io/projected/627ad113-e0fa-4637-9ac6-c71f11f43974-kube-api-access-snts4\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.547425 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.547359    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.547425 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.547382    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.547425 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.547410    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.547682 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.547449    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.547682 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.547512    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.547682 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.547550    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.548159 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.547828    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.548159 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.547922    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.548159 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.548112    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.548445 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.548175    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.548445 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.548307    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/627ad113-e0fa-4637-9ac6-c71f11f43974-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.551308 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.551241    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.551785 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.551760    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.554581 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.554557    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"]
Nov 04 02:07:37.610304 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.610223    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.610597 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.610572    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snts4\" (UniqueName: \"kubernetes.io/projected/627ad113-e0fa-4637-9ac6-c71f11f43974-kube-api-access-snts4\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r927w\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:37.650480 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:37.650451    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:38.074364 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:38.074339    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"]
Nov 04 02:07:38.075824 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:07:38.075797    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod627ad113_e0fa_4637_9ac6_c71f11f43974.slice/crio-7e7942441c5c51e6f48a3f3f33eb273b7da1ec18fcf656ea2a84fcde6b54dfd0 WatchSource:0}: Error finding container 7e7942441c5c51e6f48a3f3f33eb273b7da1ec18fcf656ea2a84fcde6b54dfd0: Status 404 returned error can't find the container with id 7e7942441c5c51e6f48a3f3f33eb273b7da1ec18fcf656ea2a84fcde6b54dfd0
Nov 04 02:07:38.077997 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:38.077959    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:07:38.078080 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:38.078018    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:07:38.078080 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:38.078049    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:07:38.993605 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:38.993562    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w" event={"ID":"627ad113-e0fa-4637-9ac6-c71f11f43974","Type":"ContainerStarted","Data":"2cbb541a50af951c6e3adda5cb62e37e78c7c5016d09a9986bff65574c2e204d"}
Nov 04 02:07:38.994019 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:38.993630    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w" event={"ID":"627ad113-e0fa-4637-9ac6-c71f11f43974","Type":"ContainerStarted","Data":"7e7942441c5c51e6f48a3f3f33eb273b7da1ec18fcf656ea2a84fcde6b54dfd0"}
Nov 04 02:07:39.157181 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:39.157111    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w" podStartSLOduration=2.157098197 podStartE2EDuration="2.157098197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:07:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:07:39.15709912 +0000 UTC m=+722.383885789" watchObservedRunningTime="2026-11-04 02:07:39.157098197 +0000 UTC m=+722.383884865"
Nov 04 02:07:39.653295 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:39.653243    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:39.653685 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:39.653658    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-r927w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Nov 04 02:07:39.653885 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:39.653858    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w" podUID="627ad113-e0fa-4637-9ac6-c71f11f43974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Nov 04 02:07:40.654411 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:40.654375    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-r927w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Nov 04 02:07:40.654991 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:40.654957    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w" podUID="627ad113-e0fa-4637-9ac6-c71f11f43974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Nov 04 02:07:41.653672 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:41.653631    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-r927w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Nov 04 02:07:41.653971 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:41.653941    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w" podUID="627ad113-e0fa-4637-9ac6-c71f11f43974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Nov 04 02:07:42.656668 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:42.656636    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-r927w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Nov 04 02:07:42.657176 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:42.656699    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w" podUID="627ad113-e0fa-4637-9ac6-c71f11f43974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Nov 04 02:07:43.655305 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:43.654637    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-r927w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Nov 04 02:07:43.655305 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:43.654702    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w" podUID="627ad113-e0fa-4637-9ac6-c71f11f43974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Nov 04 02:07:44.651784 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:44.651750    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-r927w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Nov 04 02:07:44.652247 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:44.651809    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w" podUID="627ad113-e0fa-4637-9ac6-c71f11f43974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Nov 04 02:07:45.651222 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:45.651183    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-r927w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Nov 04 02:07:45.651417 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:45.651264    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w" podUID="627ad113-e0fa-4637-9ac6-c71f11f43974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Nov 04 02:07:46.682942 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:46.682908    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:46.683417 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:46.683303    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:46.683965 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:46.683944    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:07:49.799144 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:49.799098    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bddb76fbc-vtbff"]
Nov 04 02:07:49.799835 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:49.799371    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-bddb76fbc-vtbff" podUID="cc66e1cb-8674-40ab-827a-d90415dc4dc5" containerName="authorino" containerID="cri-o://0833400d93a8cf3056baf87d68ef7777418f0278972ae335ebe40fd2146a7ebc" gracePeriod=30
Nov 04 02:07:50.046579 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:50.046550    4865 generic.go:356] "Generic (PLEG): container finished" podID="cc66e1cb-8674-40ab-827a-d90415dc4dc5" containerID="0833400d93a8cf3056baf87d68ef7777418f0278972ae335ebe40fd2146a7ebc" exitCode=0
Nov 04 02:07:50.046726 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:50.046601    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bddb76fbc-vtbff" event={"ID":"cc66e1cb-8674-40ab-827a-d90415dc4dc5","Type":"ContainerDied","Data":"0833400d93a8cf3056baf87d68ef7777418f0278972ae335ebe40fd2146a7ebc"}
Nov 04 02:07:50.046726 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:50.046635    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bddb76fbc-vtbff" event={"ID":"cc66e1cb-8674-40ab-827a-d90415dc4dc5","Type":"ContainerDied","Data":"bdaab3a05dfb87e95e494b0c57f22635982d66ca3723c605465471f596c75951"}
Nov 04 02:07:50.046726 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:50.046650    4865 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bdaab3a05dfb87e95e494b0c57f22635982d66ca3723c605465471f596c75951"
Nov 04 02:07:50.051607 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:50.051544    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bddb76fbc-vtbff"
Nov 04 02:07:50.165853 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:50.165822    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cc66e1cb-8674-40ab-827a-d90415dc4dc5-tls-cert\" (UniqueName: \"kubernetes.io/secret/cc66e1cb-8674-40ab-827a-d90415dc4dc5-tls-cert\") pod \"cc66e1cb-8674-40ab-827a-d90415dc4dc5\" (UID: \"cc66e1cb-8674-40ab-827a-d90415dc4dc5\") "
Nov 04 02:07:50.166042 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:50.165858    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc66e1cb-8674-40ab-827a-d90415dc4dc5-kube-api-access-pjbg8\" (UniqueName: \"kubernetes.io/projected/cc66e1cb-8674-40ab-827a-d90415dc4dc5-kube-api-access-pjbg8\") pod \"cc66e1cb-8674-40ab-827a-d90415dc4dc5\" (UID: \"cc66e1cb-8674-40ab-827a-d90415dc4dc5\") "
Nov 04 02:07:50.168339 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:50.168267    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc66e1cb-8674-40ab-827a-d90415dc4dc5-kube-api-access-pjbg8" pod "cc66e1cb-8674-40ab-827a-d90415dc4dc5" (UID: "cc66e1cb-8674-40ab-827a-d90415dc4dc5"). InnerVolumeSpecName "kube-api-access-pjbg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:07:50.179904 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:50.179871    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc66e1cb-8674-40ab-827a-d90415dc4dc5-tls-cert" pod "cc66e1cb-8674-40ab-827a-d90415dc4dc5" (UID: "cc66e1cb-8674-40ab-827a-d90415dc4dc5"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:07:50.267519 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:50.267487    4865 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cc66e1cb-8674-40ab-827a-d90415dc4dc5-tls-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:07:50.267519 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:50.267516    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjbg8\" (UniqueName: \"kubernetes.io/projected/cc66e1cb-8674-40ab-827a-d90415dc4dc5-kube-api-access-pjbg8\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:07:51.051040 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:51.051008    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bddb76fbc-vtbff"
Nov 04 02:07:51.221640 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:51.221592    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bddb76fbc-vtbff"]
Nov 04 02:07:51.272355 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:51.272324    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-bddb76fbc-vtbff"]
Nov 04 02:07:52.894219 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:07:52.894184    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc66e1cb-8674-40ab-827a-d90415dc4dc5" path="/var/lib/kubelet/pods/cc66e1cb-8674-40ab-827a-d90415dc4dc5/volumes"
Nov 04 02:07:58.403108 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:07:58.403066    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:08:28.403832 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:08:28.403794    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:08:45.090998 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.090961    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"]
Nov 04 02:08:45.091475 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.091353    4865 cpu_manager.go:414] "containerMap: removing container" podUID="cc66e1cb-8674-40ab-827a-d90415dc4dc5" containerName="authorino"
Nov 04 02:08:45.091475 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.091364    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc66e1cb-8674-40ab-827a-d90415dc4dc5" containerName="authorino"
Nov 04 02:08:45.091475 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.091448    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc66e1cb-8674-40ab-827a-d90415dc4dc5" containerName="authorino"
Nov 04 02:08:45.094702 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.094684    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.148178 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.148143    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-29e7e8-openshift-default-dockercfg-7ntvz\""
Nov 04 02:08:45.201372 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.201338    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"]
Nov 04 02:08:45.261668 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.261632    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.261832 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.261693    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.261832 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.261776    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.261832 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.261819    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.261970 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.261849    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wctx2\" (UniqueName: \"kubernetes.io/projected/addb3654-9be9-49e2-ba5d-9a351d932eb3-kube-api-access-wctx2\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.262017 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.261993    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.262059 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.262027    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.262059 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.262050    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/addb3654-9be9-49e2-ba5d-9a351d932eb3-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.262172 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.262155    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.362948 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.362855    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.362948 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.362902    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wctx2\" (UniqueName: \"kubernetes.io/projected/addb3654-9be9-49e2-ba5d-9a351d932eb3-kube-api-access-wctx2\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.362948 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.362936    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.363255 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.362961    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.363255 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.362991    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/addb3654-9be9-49e2-ba5d-9a351d932eb3-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.363255 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.363050    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.363255 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.363102    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.363255 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.363143    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.363255 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.363166    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.363604 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.363379    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.363604 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.363448    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.363604 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.363379    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.363757 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.363633    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.363891 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.363862    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/addb3654-9be9-49e2-ba5d-9a351d932eb3-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.365887 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.365864    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.366613 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.366589    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.425649 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.425618    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.425805 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.425782    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wctx2\" (UniqueName: \"kubernetes.io/projected/addb3654-9be9-49e2-ba5d-9a351d932eb3-kube-api-access-wctx2\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.446030 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.446005    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"]
Nov 04 02:08:45.450549 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.450533    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.510975 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.510939    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"]
Nov 04 02:08:45.566178 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.566147    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-token\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.566388 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.566208    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-workload-socket\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.566388 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.566306    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-podinfo\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.566388 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.566376    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-envoy\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.566516 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.566430    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xddq5\" (UniqueName: \"kubernetes.io/projected/8c7a83a4-3356-417d-ad04-f258a97655fd-kube-api-access-xddq5\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.566516 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.566467    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-credential-socket\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.566588 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.566562    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c7a83a4-3356-417d-ad04-f258a97655fd-istiod-ca-cert\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.566623 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.566594    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-data\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.566623 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.566612    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-workload-certs\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.573530 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.573504    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-3bf869-openshift-default-dockercfg-6f4sd\""
Nov 04 02:08:45.667428 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.667344    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xddq5\" (UniqueName: \"kubernetes.io/projected/8c7a83a4-3356-417d-ad04-f258a97655fd-kube-api-access-xddq5\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.667428 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.667391    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-credential-socket\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.667428 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.667417    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c7a83a4-3356-417d-ad04-f258a97655fd-istiod-ca-cert\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.667661 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.667437    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-data\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.667661 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.667454    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-workload-certs\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.667661 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.667610    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-token\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.667765 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.667716    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-workload-socket\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.667765 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.667742    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-podinfo\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.667838 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.667775    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-envoy\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.667962 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.667934    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-workload-certs\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.668080 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.667983    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-data\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.668080 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.668053    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-credential-socket\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.668229 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.668208    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c7a83a4-3356-417d-ad04-f258a97655fd-istiod-ca-cert\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.668412 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.668391    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-workload-socket\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.669887 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.669868    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-envoy\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.670163 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.670146    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-podinfo\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.709138 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.709106    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:45.729659 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.729639    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xddq5\" (UniqueName: \"kubernetes.io/projected/8c7a83a4-3356-417d-ad04-f258a97655fd-kube-api-access-xddq5\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.729659 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.729651    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-token\") pod \"e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") " pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.762549 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.762515    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"]
Nov 04 02:08:45.767338 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.767113    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:45.769230 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.769210    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.826311 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.825522    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-196e7e-openshift-default-dockercfg-cdj7c\""
Nov 04 02:08:45.828849 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.828496    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"]
Nov 04 02:08:45.869861 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.869176    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-data\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.869861 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.869327    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-envoy\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.869861 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.869371    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-podinfo\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.869861 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.869520    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-workload-certs\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.869861 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.869560    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42661d7a-b751-4a93-b66c-25c7d0275abf-istiod-ca-cert\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.869861 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.869593    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-workload-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.869861 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.869735    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-credential-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.869861 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.869812    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-token\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.870250 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.869869    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-556cf\" (UniqueName: \"kubernetes.io/projected/42661d7a-b751-4a93-b66c-25c7d0275abf-kube-api-access-556cf\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.970587 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.970549    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-credential-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.970771 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.970629    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-token\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.970771 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.970667    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-556cf\" (UniqueName: \"kubernetes.io/projected/42661d7a-b751-4a93-b66c-25c7d0275abf-kube-api-access-556cf\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.970771 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.970708    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-data\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.970771 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.970735    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-envoy\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.970771 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.970764    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-podinfo\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.971559 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.971508    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-workload-certs\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.971718 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.971581    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42661d7a-b751-4a93-b66c-25c7d0275abf-istiod-ca-cert\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.971718 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.971618    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-workload-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.972114 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.972055    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-workload-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.972221 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.972110    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-workload-certs\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.972270 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.972251    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-credential-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.972498 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.972480    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42661d7a-b751-4a93-b66c-25c7d0275abf-istiod-ca-cert\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.972644 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.972630    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-data\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.973780 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.973763    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-envoy\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:45.974091 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:45.974077    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-podinfo\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:46.034320 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.034250    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-token\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:46.035502 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.035474    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-556cf\" (UniqueName: \"kubernetes.io/projected/42661d7a-b751-4a93-b66c-25c7d0275abf-kube-api-access-556cf\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:46.091613 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.091580    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:46.134790 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.134747    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"]
Nov 04 02:08:46.138826 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:08:46.138790    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaddb3654_9be9_49e2_ba5d_9a351d932eb3.slice/crio-d3b9851ba4eb31bb0456eb77ddfe6f8b89e9c22aa9f44d7775a8b608fa201c10 WatchSource:0}: Error finding container d3b9851ba4eb31bb0456eb77ddfe6f8b89e9c22aa9f44d7775a8b608fa201c10: Status 404 returned error can't find the container with id d3b9851ba4eb31bb0456eb77ddfe6f8b89e9c22aa9f44d7775a8b608fa201c10
Nov 04 02:08:46.141808 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.141777    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:08:46.141895 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.141847    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:08:46.141895 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.141888    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:08:46.188522 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.188426    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"]
Nov 04 02:08:46.191863 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:08:46.191833    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c7a83a4_3356_417d_ad04_f258a97655fd.slice/crio-bf949a8c2e4288c4c87a0bb63c35786ff9bfc7751fb46cd96b1428e229112b10 WatchSource:0}: Error finding container bf949a8c2e4288c4c87a0bb63c35786ff9bfc7751fb46cd96b1428e229112b10: Status 404 returned error can't find the container with id bf949a8c2e4288c4c87a0bb63c35786ff9bfc7751fb46cd96b1428e229112b10
Nov 04 02:08:46.194641 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.194607    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:08:46.194714 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.194691    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:08:46.194751 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.194735    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:08:46.324580 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.324533    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7" event={"ID":"addb3654-9be9-49e2-ba5d-9a351d932eb3","Type":"ContainerStarted","Data":"b67d41af9c6d901f74667d0b885483d9da47f495e4cb3cf458240b525394399f"}
Nov 04 02:08:46.325033 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.324607    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7" event={"ID":"addb3654-9be9-49e2-ba5d-9a351d932eb3","Type":"ContainerStarted","Data":"d3b9851ba4eb31bb0456eb77ddfe6f8b89e9c22aa9f44d7775a8b608fa201c10"}
Nov 04 02:08:46.326143 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.326122    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9" event={"ID":"8c7a83a4-3356-417d-ad04-f258a97655fd","Type":"ContainerStarted","Data":"d77f2addb0847cfd79dcf0d565a2daa03eb8e9ef581a562488ca288589e9bf61"}
Nov 04 02:08:46.326226 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.326162    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9" event={"ID":"8c7a83a4-3356-417d-ad04-f258a97655fd","Type":"ContainerStarted","Data":"bf949a8c2e4288c4c87a0bb63c35786ff9bfc7751fb46cd96b1428e229112b10"}
Nov 04 02:08:46.449652 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.449572    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7" podStartSLOduration=1.449553458 podStartE2EDuration="1.449553458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:08:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:08:46.447709817 +0000 UTC m=+789.674496486" watchObservedRunningTime="2026-11-04 02:08:46.449553458 +0000 UTC m=+789.676340128"
Nov 04 02:08:46.468951 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.468920    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"]
Nov 04 02:08:46.470372 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:08:46.470333    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42661d7a_b751_4a93_b66c_25c7d0275abf.slice/crio-5d54084dc1cbd10261db3d0d824ab39103f6f143ead566fd4e6d3cadbc5a5e21 WatchSource:0}: Error finding container 5d54084dc1cbd10261db3d0d824ab39103f6f143ead566fd4e6d3cadbc5a5e21: Status 404 returned error can't find the container with id 5d54084dc1cbd10261db3d0d824ab39103f6f143ead566fd4e6d3cadbc5a5e21
Nov 04 02:08:46.472756 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.472693    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:08:46.472832 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.472769    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:08:46.472832 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.472814    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:08:46.574047 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.571812    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9" podStartSLOduration=1.57179509 podStartE2EDuration="1.57179509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:08:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:08:46.570482084 +0000 UTC m=+789.797268779" watchObservedRunningTime="2026-11-04 02:08:46.57179509 +0000 UTC m=+789.798581758"
Nov 04 02:08:46.709309 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.709245    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:46.767350 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:46.767233    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:47.340438 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:47.340393    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv" event={"ID":"42661d7a-b751-4a93-b66c-25c7d0275abf","Type":"ContainerStarted","Data":"0e21b016d78ba025c5f70b5861e4eb0e917053b9a02aa845a43a4467c751d740"}
Nov 04 02:08:47.340860 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:47.340496    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv" event={"ID":"42661d7a-b751-4a93-b66c-25c7d0275abf","Type":"ContainerStarted","Data":"5d54084dc1cbd10261db3d0d824ab39103f6f143ead566fd4e6d3cadbc5a5e21"}
Nov 04 02:08:47.463185 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:47.463128    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv" podStartSLOduration=2.463115863 podStartE2EDuration="2.463115863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:08:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:08:47.462698666 +0000 UTC m=+790.689485334" watchObservedRunningTime="2026-11-04 02:08:47.463115863 +0000 UTC m=+790.689902531"
Nov 04 02:08:47.715035 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:47.715003    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:47.772560 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:47.772534    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:47.935223 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:47.935189    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"]
Nov 04 02:08:47.938946 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:47.938930    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.004050 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.003964    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-24bea9-openshift-default-dockercfg-g5z8q\""
Nov 04 02:08:48.006510 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.006484    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"]
Nov 04 02:08:48.092407 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.092373    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:48.092681 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.092649    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9049038a-28f6-48ac-9971-a60bfe207660-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.093071 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.092707    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.093071 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.092746    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9049038a-28f6-48ac-9971-a60bfe207660-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.093222 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.093093    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.093397 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.093229    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.093397 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.093270    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kbl5\" (UniqueName: \"kubernetes.io/projected/9049038a-28f6-48ac-9971-a60bfe207660-kube-api-access-6kbl5\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.093397 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.093356    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9049038a-28f6-48ac-9971-a60bfe207660-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.093397 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.093393    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.093626 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.093430    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.098234 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.098203    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:48.194464 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.194434    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.194606 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.194497    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.194606 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.194520    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kbl5\" (UniqueName: \"kubernetes.io/projected/9049038a-28f6-48ac-9971-a60bfe207660-kube-api-access-6kbl5\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.194606 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.194543    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9049038a-28f6-48ac-9971-a60bfe207660-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.194606 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.194570    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.194606 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.194603    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.194873 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.194646    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9049038a-28f6-48ac-9971-a60bfe207660-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.194873 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.194683    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.194873 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.194711    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9049038a-28f6-48ac-9971-a60bfe207660-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.195038 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.195019    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.195093 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.195056    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.195262 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.195241    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.195376 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.195327    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.195761 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.195733    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9049038a-28f6-48ac-9971-a60bfe207660-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.196934 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.196911    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.197443 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.197426    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9049038a-28f6-48ac-9971-a60bfe207660-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.257002 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.256921    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9049038a-28f6-48ac-9971-a60bfe207660-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.257002 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.256996    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kbl5\" (UniqueName: \"kubernetes.io/projected/9049038a-28f6-48ac-9971-a60bfe207660-kube-api-access-6kbl5\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.345397 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.345359    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:48.345847 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.345702    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:48.345847 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.345746    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:48.346701 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.346680    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:08:48.346813 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.346742    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:08:48.346813 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.346785    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:08:48.551654 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.551579    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:48.985823 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.985685    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"]
Nov 04 02:08:48.988077 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:08:48.988052    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9049038a_28f6_48ac_9971_a60bfe207660.slice/crio-44876a270feee6fb4eeeaba2891d9012702952e1610d2e9397a6051e52064f71 WatchSource:0}: Error finding container 44876a270feee6fb4eeeaba2891d9012702952e1610d2e9397a6051e52064f71: Status 404 returned error can't find the container with id 44876a270feee6fb4eeeaba2891d9012702952e1610d2e9397a6051e52064f71
Nov 04 02:08:48.990493 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.990460    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:08:48.990602 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.990539    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:08:48.990602 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:48.990586    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:08:49.351938 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:49.351846    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d" event={"ID":"9049038a-28f6-48ac-9971-a60bfe207660","Type":"ContainerStarted","Data":"c1c4a2c059846ca23a5b2cc385a80a7d2ca0b39bc72d35628d0defff10705e1b"}
Nov 04 02:08:49.351938 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:49.351902    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d" event={"ID":"9049038a-28f6-48ac-9971-a60bfe207660","Type":"ContainerStarted","Data":"44876a270feee6fb4eeeaba2891d9012702952e1610d2e9397a6051e52064f71"}
Nov 04 02:08:49.475056 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:49.475007    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d" podStartSLOduration=2.474995006 podStartE2EDuration="2.474995006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:08:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:08:49.473795285 +0000 UTC m=+792.700581953" watchObservedRunningTime="2026-11-04 02:08:49.474995006 +0000 UTC m=+792.701781673"
Nov 04 02:08:49.551753 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:49.551719    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:50.557728 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:50.557696    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:51.361998 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:51.361967    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:51.362906 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:51.362877    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:08:53.268868 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.268834    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"]
Nov 04 02:08:53.272583 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.272565    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.358168 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.358123    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-token\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.358168 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.358173    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-data\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.358442 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.358195    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-credential-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.358442 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.358232    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-podinfo\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.358442 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.358250    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-workload-certs\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.358442 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.358269    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istiod-ca-cert\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.358442 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.358309    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-workload-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.358442 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.358333    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7xkt\" (UniqueName: \"kubernetes.io/projected/1c786c4c-0ace-4ac4-aad0-3984cb251c72-kube-api-access-r7xkt\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.358442 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.358389    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-envoy\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.367593 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.367567    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-90ad0f-openshift-default-dockercfg-nz7vm\""
Nov 04 02:08:53.381863 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.381834    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"]
Nov 04 02:08:53.460111 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.460076    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-token\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.460111 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.460110    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-data\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.460418 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.460133    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-credential-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.460418 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.460174    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-podinfo\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.460418 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.460203    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-workload-certs\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.460418 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.460229    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istiod-ca-cert\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.460418 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.460265    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-workload-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.460418 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.460339    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7xkt\" (UniqueName: \"kubernetes.io/projected/1c786c4c-0ace-4ac4-aad0-3984cb251c72-kube-api-access-r7xkt\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.460418 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.460365    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-envoy\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.460769 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.460606    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-data\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.460769 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.460647    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-credential-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.460880 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.460777    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-workload-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.460880 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.460864    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-workload-certs\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.461056 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.461035    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istiod-ca-cert\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.462457 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.462437    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-envoy\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.462763 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.462743    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-podinfo\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.545868 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.545782    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-token\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.546029 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.545882    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7xkt\" (UniqueName: \"kubernetes.io/projected/1c786c4c-0ace-4ac4-aad0-3984cb251c72-kube-api-access-r7xkt\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:53.585433 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:53.585403    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:54.050039 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:54.050012    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"]
Nov 04 02:08:54.050887 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:08:54.050862    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c786c4c_0ace_4ac4_aad0_3984cb251c72.slice/crio-5bf5d1cc722cca2c94eb4c8bc850386fb80e1c15edb17a949f6c9f092f0f34b1 WatchSource:0}: Error finding container 5bf5d1cc722cca2c94eb4c8bc850386fb80e1c15edb17a949f6c9f092f0f34b1: Status 404 returned error can't find the container with id 5bf5d1cc722cca2c94eb4c8bc850386fb80e1c15edb17a949f6c9f092f0f34b1
Nov 04 02:08:54.053272 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:54.053240    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:08:54.053363 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:54.053332    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:08:54.053400 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:54.053364    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:08:54.379488 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:54.379407    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j" event={"ID":"1c786c4c-0ace-4ac4-aad0-3984cb251c72","Type":"ContainerStarted","Data":"16fcb3dcb47a5e67eb8bc90ed64fe93d83d052f756ec655b19fcbf263fc52075"}
Nov 04 02:08:54.379488 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:54.379464    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j" event={"ID":"1c786c4c-0ace-4ac4-aad0-3984cb251c72","Type":"ContainerStarted","Data":"5bf5d1cc722cca2c94eb4c8bc850386fb80e1c15edb17a949f6c9f092f0f34b1"}
Nov 04 02:08:54.503540 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:54.503492    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j" podStartSLOduration=1.5034788749999999 podStartE2EDuration="1.503478875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:08:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:08:54.501315398 +0000 UTC m=+797.728102066" watchObservedRunningTime="2026-11-04 02:08:54.503478875 +0000 UTC m=+797.730265542"
Nov 04 02:08:54.585787 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:54.585755    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:55.591300 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:55.591242    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:55.907339 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:55.907244    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"]
Nov 04 02:08:55.910970 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:55.910950    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:08:55.964625 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:55.964557    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-3bf869-serving-cert\""
Nov 04 02:08:55.964625 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:55.964578    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hnsgt\""
Nov 04 02:08:55.964831 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:55.964723    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 02:08:55.964831 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:55.964737    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 04 02:08:55.964831 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:55.964758    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 02:08:55.987242 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:55.987215    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-maas-api-tls\") pod \"maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct\" (UID: \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:08:55.987373 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:55.987278    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-metrics-certs\") pod \"maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct\" (UID: \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:08:55.987373 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:55.987315    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrd2v\" (UniqueName: \"kubernetes.io/projected/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-kube-api-access-vrd2v\") pod \"maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct\" (UID: \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:08:55.987479 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:55.987453    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct\" (UID: \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:08:56.010170 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:56.010146    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"]
Nov 04 02:08:56.088719 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:56.088688    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-maas-api-tls\") pod \"maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct\" (UID: \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:08:56.088887 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:56.088742    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-metrics-certs\") pod \"maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct\" (UID: \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:08:56.088887 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:56.088769    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrd2v\" (UniqueName: \"kubernetes.io/projected/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-kube-api-access-vrd2v\") pod \"maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct\" (UID: \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:08:56.088887 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:56.088801    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct\" (UID: \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:08:56.088887 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:08:56.088829    4865 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-serving-cert: secret "maas-api-e2e-worker-w3-3bf869-serving-cert" not found
Nov 04 02:08:56.089076 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:08:56.088900    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-maas-api-tls podName:ee0fef40-6d80-40f1-9c16-5d92fc9759b0 nodeName:}" failed. No retries permitted until 2026-11-04 02:08:56.588885421 +0000 UTC m=+799.815672067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-maas-api-tls") pod "maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct" (UID: "ee0fef40-6d80-40f1-9c16-5d92fc9759b0") : secret "maas-api-e2e-worker-w3-3bf869-serving-cert" not found
Nov 04 02:08:56.089592 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:56.089573    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct\" (UID: \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:08:56.091166 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:56.091149    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-metrics-certs\") pod \"maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct\" (UID: \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:08:56.162237 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:56.162158    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrd2v\" (UniqueName: \"kubernetes.io/projected/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-kube-api-access-vrd2v\") pod \"maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct\" (UID: \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:08:56.389063 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:56.389036    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:56.389944 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:56.389925    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:08:56.594080 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:56.594042    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-maas-api-tls\") pod \"maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct\" (UID: \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:08:56.596578 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:56.596544    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-maas-api-tls\") pod \"maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct\" (UID: \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:08:56.824393 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:56.824361    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:08:57.205161 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:57.205137    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"]
Nov 04 02:08:57.206619 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:08:57.206585    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee0fef40_6d80_40f1_9c16_5d92fc9759b0.slice/crio-f5c6236a77581951b88e31db2522cc55eb84b3bcab3f832972c5e2cf38f27358 WatchSource:0}: Error finding container f5c6236a77581951b88e31db2522cc55eb84b3bcab3f832972c5e2cf38f27358: Status 404 returned error can't find the container with id f5c6236a77581951b88e31db2522cc55eb84b3bcab3f832972c5e2cf38f27358
Nov 04 02:08:57.395625 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:57.395589    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct" event={"ID":"ee0fef40-6d80-40f1-9c16-5d92fc9759b0","Type":"ContainerStarted","Data":"f5c6236a77581951b88e31db2522cc55eb84b3bcab3f832972c5e2cf38f27358"}
Nov 04 02:08:58.402422 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:58.402319    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct" event={"ID":"ee0fef40-6d80-40f1-9c16-5d92fc9759b0","Type":"ContainerStarted","Data":"86957a21ad5dcde15e07097e6fa8ea36ddab74f0bc5bdcab34ff8e0b572d0e05"}
Nov 04 02:08:58.402873 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:58.402481    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:08:58.403967 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:08:58.403939    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:08:58.521572 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:08:58.521525    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct" podStartSLOduration=2.6880630009999997 podStartE2EDuration="3.521513651s" totalImagesPullingTime="833.45065ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:08:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:08:58.520978601 +0000 UTC m=+801.747765269" watchObservedRunningTime="2026-11-04 02:08:58.521513651 +0000 UTC m=+801.748300319"
Nov 04 02:09:01.161245 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:01.161213    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"]
Nov 04 02:09:01.164993 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:01.164962    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:09:01.243359 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:01.243323    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f207e785-019b-4488-8ef4-8a0f56f8134d-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk\" (UID: \"f207e785-019b-4488-8ef4-8a0f56f8134d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:09:01.243518 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:01.243389    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f207e785-019b-4488-8ef4-8a0f56f8134d-metrics-certs\") pod \"maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk\" (UID: \"f207e785-019b-4488-8ef4-8a0f56f8134d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:09:01.243518 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:01.243502    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f207e785-019b-4488-8ef4-8a0f56f8134d-maas-api-tls\") pod \"maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk\" (UID: \"f207e785-019b-4488-8ef4-8a0f56f8134d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:09:01.243654 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:01.243555    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pv7w\" (UniqueName: \"kubernetes.io/projected/f207e785-019b-4488-8ef4-8a0f56f8134d-kube-api-access-7pv7w\") pod \"maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk\" (UID: \"f207e785-019b-4488-8ef4-8a0f56f8134d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:09:01.271702 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:01.271673    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-196e7e-serving-cert\""
Nov 04 02:09:01.273690 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:01.273668    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"]
Nov 04 02:09:01.344696 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:01.344661    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f207e785-019b-4488-8ef4-8a0f56f8134d-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk\" (UID: \"f207e785-019b-4488-8ef4-8a0f56f8134d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:09:01.344841 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:01.344715    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f207e785-019b-4488-8ef4-8a0f56f8134d-metrics-certs\") pod \"maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk\" (UID: \"f207e785-019b-4488-8ef4-8a0f56f8134d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:09:01.344841 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:01.344777    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f207e785-019b-4488-8ef4-8a0f56f8134d-maas-api-tls\") pod \"maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk\" (UID: \"f207e785-019b-4488-8ef4-8a0f56f8134d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:09:01.344841 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:01.344823    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pv7w\" (UniqueName: \"kubernetes.io/projected/f207e785-019b-4488-8ef4-8a0f56f8134d-kube-api-access-7pv7w\") pod \"maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk\" (UID: \"f207e785-019b-4488-8ef4-8a0f56f8134d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:09:01.344971 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:09:01.344901    4865 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-serving-cert: secret "maas-api-e2e-worker-w4-196e7e-serving-cert" not found
Nov 04 02:09:01.344971 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:09:01.344969    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f207e785-019b-4488-8ef4-8a0f56f8134d-maas-api-tls podName:f207e785-019b-4488-8ef4-8a0f56f8134d nodeName:}" failed. No retries permitted until 2026-11-04 02:09:01.844951382 +0000 UTC m=+805.071738028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f207e785-019b-4488-8ef4-8a0f56f8134d-maas-api-tls") pod "maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk" (UID: "f207e785-019b-4488-8ef4-8a0f56f8134d") : secret "maas-api-e2e-worker-w4-196e7e-serving-cert" not found
Nov 04 02:09:01.345509 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:01.345484    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f207e785-019b-4488-8ef4-8a0f56f8134d-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk\" (UID: \"f207e785-019b-4488-8ef4-8a0f56f8134d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:09:01.347127 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:01.347108    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f207e785-019b-4488-8ef4-8a0f56f8134d-metrics-certs\") pod \"maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk\" (UID: \"f207e785-019b-4488-8ef4-8a0f56f8134d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:09:01.427937 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:01.427816    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pv7w\" (UniqueName: \"kubernetes.io/projected/f207e785-019b-4488-8ef4-8a0f56f8134d-kube-api-access-7pv7w\") pod \"maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk\" (UID: \"f207e785-019b-4488-8ef4-8a0f56f8134d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:09:01.850460 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:01.850421    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f207e785-019b-4488-8ef4-8a0f56f8134d-maas-api-tls\") pod \"maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk\" (UID: \"f207e785-019b-4488-8ef4-8a0f56f8134d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:09:01.852691 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:01.852664    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f207e785-019b-4488-8ef4-8a0f56f8134d-maas-api-tls\") pod \"maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk\" (UID: \"f207e785-019b-4488-8ef4-8a0f56f8134d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:09:02.077517 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:02.077478    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:09:02.454199 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:02.454175    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"]
Nov 04 02:09:02.455882 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:09:02.455858    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf207e785_019b_4488_8ef4_8a0f56f8134d.slice/crio-b95706c050ce02791c5a498f6b0284685709b93989cd00e3e533f0cf1a90aaea WatchSource:0}: Error finding container b95706c050ce02791c5a498f6b0284685709b93989cd00e3e533f0cf1a90aaea: Status 404 returned error can't find the container with id b95706c050ce02791c5a498f6b0284685709b93989cd00e3e533f0cf1a90aaea
Nov 04 02:09:03.443556 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:03.443522    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk" event={"ID":"f207e785-019b-4488-8ef4-8a0f56f8134d","Type":"ContainerStarted","Data":"8f4d31e8f43522fba2f7d5b78e4765210198b5624df103e0ceb0ee19c5627252"}
Nov 04 02:09:03.443728 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:03.443575    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk" event={"ID":"f207e785-019b-4488-8ef4-8a0f56f8134d","Type":"ContainerStarted","Data":"b95706c050ce02791c5a498f6b0284685709b93989cd00e3e533f0cf1a90aaea"}
Nov 04 02:09:03.443728 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:03.443627    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:09:03.584937 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:03.584819    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk" podStartSLOduration=1.7204790669999999 podStartE2EDuration="2.584797949s" totalImagesPullingTime="864.318882ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:09:03.582337109 +0000 UTC m=+806.809123781" watchObservedRunningTime="2026-11-04 02:09:03.584797949 +0000 UTC m=+806.811584620"
Nov 04 02:09:04.422837 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:04.422796    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:09:04.635346 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:04.635314    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:09:09.465188 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:09.465153    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:09:13.548355 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:13.548320    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:09:14.510375 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:14.510343    4865 generic.go:356] "Generic (PLEG): container finished" podID="8c7a83a4-3356-417d-ad04-f258a97655fd" containerID="d77f2addb0847cfd79dcf0d565a2daa03eb8e9ef581a562488ca288589e9bf61" exitCode=0
Nov 04 02:09:14.510375 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:14.510369    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9" event={"ID":"8c7a83a4-3356-417d-ad04-f258a97655fd","Type":"ContainerDied","Data":"d77f2addb0847cfd79dcf0d565a2daa03eb8e9ef581a562488ca288589e9bf61"}
Nov 04 02:09:14.510839 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:14.510824    4865 scope.go:122] "RemoveContainer" containerID="d77f2addb0847cfd79dcf0d565a2daa03eb8e9ef581a562488ca288589e9bf61"
Nov 04 02:09:14.512217 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:14.512193    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:09:14.512292 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:14.512265    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:09:14.512342 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:14.512328    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:09:15.518935 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:15.518899    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9" event={"ID":"8c7a83a4-3356-417d-ad04-f258a97655fd","Type":"ContainerStarted","Data":"ee3630ac199e04f5e431e39987e58e025aec01ad876ebb58362a6e05c807548c"}
Nov 04 02:09:15.768495 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:15.768463    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:09:15.768838 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:15.768817    4865 patch_prober.go:29] interesting pod/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 04 02:09:15.769021 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:15.768963    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9" podUID="8c7a83a4-3356-417d-ad04-f258a97655fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 04 02:09:16.768306 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:16.767697    4865 patch_prober.go:29] interesting pod/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 04 02:09:16.768306 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:16.767774    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9" podUID="8c7a83a4-3356-417d-ad04-f258a97655fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 04 02:09:17.780323 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:17.779423    4865 patch_prober.go:29] interesting pod/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 04 02:09:17.780323 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:17.779506    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9" podUID="8c7a83a4-3356-417d-ad04-f258a97655fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 04 02:09:18.348419 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:18.348378    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:09:18.767721 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:18.767682    4865 patch_prober.go:29] interesting pod/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 04 02:09:18.768016 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:18.767988    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9" podUID="8c7a83a4-3356-417d-ad04-f258a97655fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 04 02:09:19.768064 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:19.768018    4865 patch_prober.go:29] interesting pod/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 04 02:09:19.768465 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:19.768091    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9" podUID="8c7a83a4-3356-417d-ad04-f258a97655fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 04 02:09:20.767550 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:20.767515    4865 patch_prober.go:29] interesting pod/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 04 02:09:20.767711 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:20.767565    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9" podUID="8c7a83a4-3356-417d-ad04-f258a97655fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 04 02:09:21.549256 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.549225    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"]
Nov 04 02:09:21.552926 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.552906    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:09:21.657091 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.656694    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-metrics-certs\") pod \"maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt\" (UID: \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:09:21.657091 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.656796    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt\" (UID: \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:09:21.657091 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.656860    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsrvh\" (UniqueName: \"kubernetes.io/projected/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-kube-api-access-hsrvh\") pod \"maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt\" (UID: \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:09:21.657091 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.656926    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-maas-api-tls\") pod \"maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt\" (UID: \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:09:21.665536 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.665504    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-90ad0f-serving-cert\""
Nov 04 02:09:21.667792 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.667771    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"]
Nov 04 02:09:21.758448 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.758399    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-metrics-certs\") pod \"maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt\" (UID: \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:09:21.758611 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.758503    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt\" (UID: \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:09:21.758611 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.758562    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsrvh\" (UniqueName: \"kubernetes.io/projected/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-kube-api-access-hsrvh\") pod \"maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt\" (UID: \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:09:21.758683 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.758630    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-maas-api-tls\") pod \"maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt\" (UID: \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:09:21.758834 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:09:21.758815    4865 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-serving-cert: secret "maas-api-e2e-worker-w1-90ad0f-serving-cert" not found
Nov 04 02:09:21.758929 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:09:21.758903    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-maas-api-tls podName:a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1 nodeName:}" failed. No retries permitted until 2026-11-04 02:09:22.258873187 +0000 UTC m=+825.485659857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-maas-api-tls") pod "maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt" (UID: "a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1") : secret "maas-api-e2e-worker-w1-90ad0f-serving-cert" not found
Nov 04 02:09:21.759376 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.759349    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt\" (UID: \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:09:21.761169 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.761142    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-metrics-certs\") pod \"maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt\" (UID: \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:09:21.773160 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.773139    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:09:21.774432 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.774410    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:09:21.835275 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.831619    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsrvh\" (UniqueName: \"kubernetes.io/projected/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-kube-api-access-hsrvh\") pod \"maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt\" (UID: \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:09:21.857648 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:21.857617    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:09:22.277854 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:22.277797    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-maas-api-tls\") pod \"maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt\" (UID: \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:09:22.281250 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:22.281215    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-maas-api-tls\") pod \"maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt\" (UID: \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:09:22.403448 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:22.403413    4865 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:09:22.403613 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:22.403482    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d" podUID="9049038a-28f6-48ac-9971-a60bfe207660" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:09:22.466364 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:22.466036    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:09:22.857309 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:22.854189    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"]
Nov 04 02:09:22.862456 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:09:22.862409    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda82e2dda_81f8_4c69_8f84_6a2a6fafb9a1.slice/crio-5853ffa22200d906f0d9cf5a141fd5b074ebb2a549296ced9a412e4b6e1aba10 WatchSource:0}: Error finding container 5853ffa22200d906f0d9cf5a141fd5b074ebb2a549296ced9a412e4b6e1aba10: Status 404 returned error can't find the container with id 5853ffa22200d906f0d9cf5a141fd5b074ebb2a549296ced9a412e4b6e1aba10
Nov 04 02:09:23.611116 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:23.611076    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt" event={"ID":"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1","Type":"ContainerStarted","Data":"5853ffa22200d906f0d9cf5a141fd5b074ebb2a549296ced9a412e4b6e1aba10"}
Nov 04 02:09:24.622083 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:24.620865    4865 generic.go:356] "Generic (PLEG): container finished" podID="42661d7a-b751-4a93-b66c-25c7d0275abf" containerID="0e21b016d78ba025c5f70b5861e4eb0e917053b9a02aa845a43a4467c751d740" exitCode=0
Nov 04 02:09:24.622083 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:24.620977    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv" event={"ID":"42661d7a-b751-4a93-b66c-25c7d0275abf","Type":"ContainerDied","Data":"0e21b016d78ba025c5f70b5861e4eb0e917053b9a02aa845a43a4467c751d740"}
Nov 04 02:09:24.622083 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:24.621412    4865 scope.go:122] "RemoveContainer" containerID="0e21b016d78ba025c5f70b5861e4eb0e917053b9a02aa845a43a4467c751d740"
Nov 04 02:09:24.625544 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:24.625239    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt" event={"ID":"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1","Type":"ContainerStarted","Data":"bd5b7e1f5ed7b42c8a667e757fc11d81cfcc6d6ab9e1564b86c83f4f0c4cf945"}
Nov 04 02:09:24.625953 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:24.625920    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:09:24.626056 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:24.626016    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:09:24.626130 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:24.626093    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:09:24.628063 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:24.628028    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d" event={"ID":"9049038a-28f6-48ac-9971-a60bfe207660","Type":"ContainerDied","Data":"c1c4a2c059846ca23a5b2cc385a80a7d2ca0b39bc72d35628d0defff10705e1b"}
Nov 04 02:09:24.628480 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:24.628422    4865 scope.go:122] "RemoveContainer" containerID="c1c4a2c059846ca23a5b2cc385a80a7d2ca0b39bc72d35628d0defff10705e1b"
Nov 04 02:09:24.629359 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:24.629330    4865 generic.go:356] "Generic (PLEG): container finished" podID="9049038a-28f6-48ac-9971-a60bfe207660" containerID="c1c4a2c059846ca23a5b2cc385a80a7d2ca0b39bc72d35628d0defff10705e1b" exitCode=0
Nov 04 02:09:24.629834 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:24.629801    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:09:24.629918 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:24.629903    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:09:24.629984 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:24.629968    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:09:24.917652 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:24.917559    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt" podStartSLOduration=3.075981329 podStartE2EDuration="3.917546569s" totalImagesPullingTime="841.56524ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:09:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:09:24.916695418 +0000 UTC m=+828.143482087" watchObservedRunningTime="2026-11-04 02:09:24.917546569 +0000 UTC m=+828.144333236"
Nov 04 02:09:25.665305 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:25.661892    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv" event={"ID":"42661d7a-b751-4a93-b66c-25c7d0275abf","Type":"ContainerStarted","Data":"3857d37b28639e897ea46867f520dd1eafc8efa0c88a57340ce4afd237f5faeb"}
Nov 04 02:09:25.678311 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:25.677604    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d" event={"ID":"9049038a-28f6-48ac-9971-a60bfe207660","Type":"ContainerStarted","Data":"f5d9f2f3a47cd2542453e960282e29d2d46a385645d3276450f47f1b565e833a"}
Nov 04 02:09:25.678311 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:25.677668    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:09:26.094781 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:26.094742    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:09:26.095103 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:26.095081    4865 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Nov 04 02:09:26.095329 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:26.095302    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv" podUID="42661d7a-b751-4a93-b66c-25c7d0275abf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Nov 04 02:09:26.555770 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:26.555732    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:09:26.556212 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:26.556077    4865 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Nov 04 02:09:26.556442 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:26.556417    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d" podUID="9049038a-28f6-48ac-9971-a60bfe207660" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Nov 04 02:09:27.103308 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:27.099699    4865 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Nov 04 02:09:27.103308 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:27.099760    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv" podUID="42661d7a-b751-4a93-b66c-25c7d0275abf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Nov 04 02:09:27.552841 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:27.552807    4865 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Nov 04 02:09:27.553203 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:27.553152    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d" podUID="9049038a-28f6-48ac-9971-a60bfe207660" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Nov 04 02:09:28.092213 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:28.092164    4865 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Nov 04 02:09:28.092399 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:28.092246    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv" podUID="42661d7a-b751-4a93-b66c-25c7d0275abf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Nov 04 02:09:28.490976 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:09:28.490915    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:09:28.553122 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:28.552675    4865 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Nov 04 02:09:28.553122 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:28.552736    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d" podUID="9049038a-28f6-48ac-9971-a60bfe207660" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Nov 04 02:09:29.092392 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:29.092355    4865 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Nov 04 02:09:29.092567 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:29.092409    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv" podUID="42661d7a-b751-4a93-b66c-25c7d0275abf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Nov 04 02:09:29.560307 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:29.556371    4865 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Nov 04 02:09:29.560307 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:29.556435    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d" podUID="9049038a-28f6-48ac-9971-a60bfe207660" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Nov 04 02:09:30.097330 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:30.092001    4865 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Nov 04 02:09:30.097330 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:30.092065    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv" podUID="42661d7a-b751-4a93-b66c-25c7d0275abf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Nov 04 02:09:30.563751 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:30.563711    4865 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Nov 04 02:09:30.564367 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:30.564335    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d" podUID="9049038a-28f6-48ac-9971-a60bfe207660" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Nov 04 02:09:31.246307 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:31.245738    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:09:31.252314 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:31.247592    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:09:31.253839 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:31.253812    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:09:31.566435 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:31.566349    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:09:31.567585 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:31.567556    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:09:31.568861 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:31.568498    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:09:31.707907 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:31.707880    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:09:32.760147 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:32.760115    4865 generic.go:356] "Generic (PLEG): container finished" podID="addb3654-9be9-49e2-ba5d-9a351d932eb3" containerID="b67d41af9c6d901f74667d0b885483d9da47f495e4cb3cf458240b525394399f" exitCode=0
Nov 04 02:09:32.760734 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:32.760201    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7" event={"ID":"addb3654-9be9-49e2-ba5d-9a351d932eb3","Type":"ContainerDied","Data":"b67d41af9c6d901f74667d0b885483d9da47f495e4cb3cf458240b525394399f"}
Nov 04 02:09:32.760813 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:32.760760    4865 scope.go:122] "RemoveContainer" containerID="b67d41af9c6d901f74667d0b885483d9da47f495e4cb3cf458240b525394399f"
Nov 04 02:09:32.761974 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:32.761946    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:09:32.762081 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:32.762016    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:09:32.762081 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:32.762051    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:09:33.345473 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:33.345425    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:09:33.785535 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:33.785240    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7" event={"ID":"addb3654-9be9-49e2-ba5d-9a351d932eb3","Type":"ContainerStarted","Data":"354a92e997367e9ee849e994827f7a37eec86b1c82a8f8fdb067fd7a40382555"}
Nov 04 02:09:34.709865 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:34.709831    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:09:34.710810 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:34.710787    4865 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Nov 04 02:09:34.710869 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:34.710830    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7" podUID="addb3654-9be9-49e2-ba5d-9a351d932eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Nov 04 02:09:35.710021 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:35.709988    4865 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Nov 04 02:09:35.710540 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:35.710511    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7" podUID="addb3654-9be9-49e2-ba5d-9a351d932eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Nov 04 02:09:36.710043 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:36.710008    4865 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Nov 04 02:09:36.710474 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:36.710074    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7" podUID="addb3654-9be9-49e2-ba5d-9a351d932eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Nov 04 02:09:37.710053 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:37.710009    4865 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Nov 04 02:09:37.710475 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:37.710086    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7" podUID="addb3654-9be9-49e2-ba5d-9a351d932eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Nov 04 02:09:38.317363 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:38.317323    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:09:38.710440 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:38.710398    4865 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Nov 04 02:09:38.710936 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:38.710450    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7" podUID="addb3654-9be9-49e2-ba5d-9a351d932eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Nov 04 02:09:39.714114 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:39.714076    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:09:39.714586 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:39.714382    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:09:39.715215 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:39.715190    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:09:40.822902 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:40.822865    4865 generic.go:356] "Generic (PLEG): container finished" podID="1c786c4c-0ace-4ac4-aad0-3984cb251c72" containerID="16fcb3dcb47a5e67eb8bc90ed64fe93d83d052f756ec655b19fcbf263fc52075" exitCode=0
Nov 04 02:09:40.823464 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:40.822941    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j" event={"ID":"1c786c4c-0ace-4ac4-aad0-3984cb251c72","Type":"ContainerDied","Data":"16fcb3dcb47a5e67eb8bc90ed64fe93d83d052f756ec655b19fcbf263fc52075"}
Nov 04 02:09:40.823588 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:40.823571    4865 scope.go:122] "RemoveContainer" containerID="16fcb3dcb47a5e67eb8bc90ed64fe93d83d052f756ec655b19fcbf263fc52075"
Nov 04 02:09:40.824741 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:40.824713    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:09:40.824825 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:40.824776    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:09:40.824825 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:40.824807    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:09:41.389712 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:41.389681    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:09:41.840468 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:41.833397    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j" event={"ID":"1c786c4c-0ace-4ac4-aad0-3984cb251c72","Type":"ContainerStarted","Data":"ebea8f67a82330104fc86273028fa40cc71b103afa1f169eb9a0c2fc2a93e0a0"}
Nov 04 02:09:42.586600 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:42.586561    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:09:42.587633 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:42.587611    4865 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Nov 04 02:09:42.587822 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:42.587676    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j" podUID="1c786c4c-0ace-4ac4-aad0-3984cb251c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Nov 04 02:09:43.585891 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:43.585853    4865 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Nov 04 02:09:43.586346 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:43.585913    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j" podUID="1c786c4c-0ace-4ac4-aad0-3984cb251c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Nov 04 02:09:44.586906 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:44.586824    4865 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Nov 04 02:09:44.587343 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:44.586903    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j" podUID="1c786c4c-0ace-4ac4-aad0-3984cb251c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Nov 04 02:09:45.586859 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:45.586820    4865 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Nov 04 02:09:45.587042 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:45.586878    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j" podUID="1c786c4c-0ace-4ac4-aad0-3984cb251c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Nov 04 02:09:46.586165 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:46.586122    4865 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Nov 04 02:09:46.586388 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:46.586184    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j" podUID="1c786c4c-0ace-4ac4-aad0-3984cb251c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Nov 04 02:09:47.590351 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:47.590321    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:09:47.590728 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:47.590597    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:09:47.591373 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:47.591353    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:09:49.425040 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:09:49.425006    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:09:58.405077 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:09:58.405040    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:10:12.361092 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:12.361053    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:10:25.212253 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.212218    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"]
Nov 04 02:10:25.216217 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.216199    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.246756 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.246721    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-model-cache\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.246924 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.246854    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-tmp-dir\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.246994 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.246933    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-home\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.247053 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.247003    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-dshm\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.247113 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.247082    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dfcdafc-3763-472d-b34f-1783a66e838c-tls-certs\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.247222 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.247194    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqfld\" (UniqueName: \"kubernetes.io/projected/4dfcdafc-3763-472d-b34f-1783a66e838c-kube-api-access-lqfld\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.322362 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.322327    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-90ad0f\"/\"openshift-service-ca.crt\""
Nov 04 02:10:25.322555 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.322452    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-90ad0f\"/\"simulator-w1-90ad0f-kserve-self-signed-certs\""
Nov 04 02:10:25.322605 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.322563    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-90ad0f\"/\"kube-root-ca.crt\""
Nov 04 02:10:25.322658 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.322615    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-90ad0f\"/\"default-dockercfg-zrd8x\""
Nov 04 02:10:25.324407 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.324387    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"]
Nov 04 02:10:25.356655 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.356610    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-tmp-dir\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.356870 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.356677    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-home\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.356870 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.356721    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-dshm\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.356870 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.356801    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dfcdafc-3763-472d-b34f-1783a66e838c-tls-certs\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.356870 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.356855    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqfld\" (UniqueName: \"kubernetes.io/projected/4dfcdafc-3763-472d-b34f-1783a66e838c-kube-api-access-lqfld\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.357138 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.356913    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-model-cache\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.359537 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.359461    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-home\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.359537 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.359467    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-tmp-dir\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.359783 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.359544    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-model-cache\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.368325 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.368226    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-dshm\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.369547 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.369499    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dfcdafc-3763-472d-b34f-1783a66e838c-tls-certs\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.421459 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.421417    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqfld\" (UniqueName: \"kubernetes.io/projected/4dfcdafc-3763-472d-b34f-1783a66e838c-kube-api-access-lqfld\") pod \"simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") " pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.527573 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.527482    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:25.903998 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:25.903967    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"]
Nov 04 02:10:25.905397 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:10:25.905374    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4dfcdafc_3763_472d_b34f_1783a66e838c.slice/crio-78916cc8350ce3a13e4395f7429439981a452fdb417ac961b2ad84afb11dd6ec WatchSource:0}: Error finding container 78916cc8350ce3a13e4395f7429439981a452fdb417ac961b2ad84afb11dd6ec: Status 404 returned error can't find the container with id 78916cc8350ce3a13e4395f7429439981a452fdb417ac961b2ad84afb11dd6ec
Nov 04 02:10:26.120062 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:26.120028    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v" event={"ID":"4dfcdafc-3763-472d-b34f-1783a66e838c","Type":"ContainerStarted","Data":"2cc93dbded9dbbd71ea85d8c1bb18370fddc73afad24b1c9c9085d2a9e3e86d4"}
Nov 04 02:10:26.120062 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:26.120068    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v" event={"ID":"4dfcdafc-3763-472d-b34f-1783a66e838c","Type":"ContainerStarted","Data":"78916cc8350ce3a13e4395f7429439981a452fdb417ac961b2ad84afb11dd6ec"}
Nov 04 02:10:26.239562 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:26.239507    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v" podStartSLOduration=1.239493182 podStartE2EDuration="1.239493182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:26.238892439 +0000 UTC m=+889.465679153" watchObservedRunningTime="2026-11-04 02:10:26.239493182 +0000 UTC m=+889.466279849"
Nov 04 02:10:26.528645 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:26.528572    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:28.404008 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:10:28.403971    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:10:33.914198 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:33.914163    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:10:35.527689 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:35.527659    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:35.540057 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:35.540029    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:36.181880 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:36.181845    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:10:36.971318 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:36.971267    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/2.log"
Nov 04 02:10:36.974756 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:36.974730    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/2.log"
Nov 04 02:10:36.982697 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:36.982679    4865 scope.go:122] "RemoveContainer" containerID="0833400d93a8cf3056baf87d68ef7777418f0278972ae335ebe40fd2146a7ebc"
Nov 04 02:10:49.303985 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:10:49.303949    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:10:58.403850 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:10:58.403815    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:11:03.793931 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.793895    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"]
Nov 04 02:11:03.798194 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.798173    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:03.843225 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.843188    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-tmp-dir\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:03.843434 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.843245    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-home\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:03.843483 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.843422    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwjrt\" (UniqueName: \"kubernetes.io/projected/43d28dad-923c-40d5-96e9-10681d488d0b-kube-api-access-pwjrt\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:03.843522 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.843477    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43d28dad-923c-40d5-96e9-10681d488d0b-tls-certs\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:03.843565 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.843533    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-model-cache\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:03.843717 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.843698    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-dshm\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:03.853033 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.853001    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-24bea9\"/\"kube-root-ca.crt\""
Nov 04 02:11:03.853272 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.853121    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-24bea9\"/\"openshift-service-ca.crt\""
Nov 04 02:11:03.904074 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.904029    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-24bea9\"/\"premium-w0-24bea9-kserve-self-signed-certs\""
Nov 04 02:11:03.904074 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.904035    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-24bea9\"/\"default-dockercfg-cz6zr\""
Nov 04 02:11:03.906425 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.906398    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"]
Nov 04 02:11:03.955959 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.954743    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-dshm\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:03.955959 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.954805    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-tmp-dir\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:03.955959 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.954863    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-home\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:03.955959 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.954924    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwjrt\" (UniqueName: \"kubernetes.io/projected/43d28dad-923c-40d5-96e9-10681d488d0b-kube-api-access-pwjrt\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:03.955959 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.954960    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43d28dad-923c-40d5-96e9-10681d488d0b-tls-certs\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:03.955959 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.955016    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-model-cache\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:03.955959 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.955630    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-model-cache\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:03.967386 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.966120    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-home\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:03.967386 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.966425    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-tmp-dir\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:03.967386 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.967332    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-dshm\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:03.975035 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:03.975002    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43d28dad-923c-40d5-96e9-10681d488d0b-tls-certs\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:04.030117 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:04.030074    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwjrt\" (UniqueName: \"kubernetes.io/projected/43d28dad-923c-40d5-96e9-10681d488d0b-kube-api-access-pwjrt\") pod \"premium-w0-24bea9-kserve-65b97b57bf-hvlc4\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") " pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:04.111020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:04.110935    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:04.490748 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:04.490725    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"]
Nov 04 02:11:04.492473 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:11:04.492443    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43d28dad_923c_40d5_96e9_10681d488d0b.slice/crio-e00517f6e75fce2ae028d79f98c634f3d36adaa09ba650930a9d1df91afe7f6f WatchSource:0}: Error finding container e00517f6e75fce2ae028d79f98c634f3d36adaa09ba650930a9d1df91afe7f6f: Status 404 returned error can't find the container with id e00517f6e75fce2ae028d79f98c634f3d36adaa09ba650930a9d1df91afe7f6f
Nov 04 02:11:05.322997 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:05.322953    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4" event={"ID":"43d28dad-923c-40d5-96e9-10681d488d0b","Type":"ContainerStarted","Data":"a683862d66fd964399d40af78f1c8de94c89b13346d55a62ba54cb57e9a81dc9"}
Nov 04 02:11:05.322997 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:05.322999    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4" event={"ID":"43d28dad-923c-40d5-96e9-10681d488d0b","Type":"ContainerStarted","Data":"e00517f6e75fce2ae028d79f98c634f3d36adaa09ba650930a9d1df91afe7f6f"}
Nov 04 02:11:05.442573 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:05.442521    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4" podStartSLOduration=2.44250744 podStartE2EDuration="2.44250744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:11:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:11:05.441452889 +0000 UTC m=+928.668239556" watchObservedRunningTime="2026-11-04 02:11:05.44250744 +0000 UTC m=+928.669294133"
Nov 04 02:11:06.111673 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:06.111624    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:11.019068 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:11.019033    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:11:14.111463 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:14.111427    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:14.127313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:14.124592    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:14.383396 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:14.383297    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:11:28.403055 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:11:28.403016    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:11:35.649250 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:35.649212    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:11:37.243681 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.243641    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"]
Nov 04 02:11:37.247490 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.247467    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.353397 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.353364    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-196e7e\"/\"kube-root-ca.crt\""
Nov 04 02:11:37.353583 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.353491    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-196e7e\"/\"openshift-service-ca.crt\""
Nov 04 02:11:37.353583 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.353491    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-196e7e\"/\"premium-w4-196e7e-kserve-self-signed-certs\""
Nov 04 02:11:37.353999 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.353967    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-196e7e\"/\"default-dockercfg-758fc\""
Nov 04 02:11:37.357825 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.357792    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"]
Nov 04 02:11:37.406094 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.406052    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-dshm\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.406337 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.406195    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-tls-certs\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.406424 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.406398    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nw45\" (UniqueName: \"kubernetes.io/projected/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-kube-api-access-6nw45\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.406518 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.406487    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-model-cache\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.406672 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.406644    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-tmp-dir\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.406738 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.406722    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-home\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.507487 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.507389    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nw45\" (UniqueName: \"kubernetes.io/projected/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-kube-api-access-6nw45\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.507487 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.507470    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-model-cache\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.507733 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.507549    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-tmp-dir\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.507733 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.507600    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-home\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.507733 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.507638    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-dshm\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.507733 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.507674    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-tls-certs\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.507974 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.507948    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-model-cache\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.508037 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.507958    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-tmp-dir\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.508037 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.508011    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-home\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.509912 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.509887    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-dshm\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.510112 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.510094    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-tls-certs\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.568772 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.568743    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nw45\" (UniqueName: \"kubernetes.io/projected/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-kube-api-access-6nw45\") pod \"premium-w4-196e7e-kserve-5fb85f5b67-d2wkj\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") " pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:37.860257 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:37.860178    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:38.234203 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:38.234172    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"]
Nov 04 02:11:38.235172 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:11:38.235146    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podead7c7c3_0931_4bde_884b_69aa8dbd28e0.slice/crio-0be545a1acc87699722e1eaf9f16b090429f7696b6fc4c42e846b1a69c3061a7 WatchSource:0}: Error finding container 0be545a1acc87699722e1eaf9f16b090429f7696b6fc4c42e846b1a69c3061a7: Status 404 returned error can't find the container with id 0be545a1acc87699722e1eaf9f16b090429f7696b6fc4c42e846b1a69c3061a7
Nov 04 02:11:38.507386 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:38.507304    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj" event={"ID":"ead7c7c3-0931-4bde-884b-69aa8dbd28e0","Type":"ContainerStarted","Data":"c9a3a794712ae76f7082f0c4a1368f9abcd6d6d85b5eb74f0523ad92e8143956"}
Nov 04 02:11:38.507386 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:38.507347    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj" event={"ID":"ead7c7c3-0931-4bde-884b-69aa8dbd28e0","Type":"ContainerStarted","Data":"0be545a1acc87699722e1eaf9f16b090429f7696b6fc4c42e846b1a69c3061a7"}
Nov 04 02:11:38.627327 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:38.627258    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj" podStartSLOduration=1.6272440860000001 podStartE2EDuration="1.627244086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:11:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:11:38.625767435 +0000 UTC m=+961.852554103" watchObservedRunningTime="2026-11-04 02:11:38.627244086 +0000 UTC m=+961.854030753"
Nov 04 02:11:38.860389 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:38.860275    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:47.860883 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:47.860850    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:47.873922 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:47.873898    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:48.572039 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:48.572004    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:11:58.303989 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:11:58.303949    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:11:58.403856 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:11:58.403813    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:12:28.404052 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:12:28.404016    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:12:32.123838 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:32.123799    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:12:37.312786 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:37.312746    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"]
Nov 04 02:12:37.313314 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:37.312960    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w" podUID="627ad113-e0fa-4637-9ac6-c71f11f43974" containerName="istio-proxy" containerID="cri-o://2cbb541a50af951c6e3adda5cb62e37e78c7c5016d09a9986bff65574c2e204d" gracePeriod=30
Nov 04 02:12:42.577142 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.577078    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:12:42.611439 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.611408    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-token\" (UniqueName: \"kubernetes.io/projected/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-token\") pod \"627ad113-e0fa-4637-9ac6-c71f11f43974\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") "
Nov 04 02:12:42.611575 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.611451    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-podinfo\") pod \"627ad113-e0fa-4637-9ac6-c71f11f43974\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") "
Nov 04 02:12:42.611575 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.611478    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/627ad113-e0fa-4637-9ac6-c71f11f43974-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/627ad113-e0fa-4637-9ac6-c71f11f43974-istiod-ca-cert\") pod \"627ad113-e0fa-4637-9ac6-c71f11f43974\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") "
Nov 04 02:12:42.611575 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.611512    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-workload-certs\") pod \"627ad113-e0fa-4637-9ac6-c71f11f43974\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") "
Nov 04 02:12:42.611575 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.611550    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-envoy\") pod \"627ad113-e0fa-4637-9ac6-c71f11f43974\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") "
Nov 04 02:12:42.611787 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.611585    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/627ad113-e0fa-4637-9ac6-c71f11f43974-kube-api-access-snts4\" (UniqueName: \"kubernetes.io/projected/627ad113-e0fa-4637-9ac6-c71f11f43974-kube-api-access-snts4\") pod \"627ad113-e0fa-4637-9ac6-c71f11f43974\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") "
Nov 04 02:12:42.611787 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.611606    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-workload-socket\") pod \"627ad113-e0fa-4637-9ac6-c71f11f43974\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") "
Nov 04 02:12:42.611787 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.611643    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-data\") pod \"627ad113-e0fa-4637-9ac6-c71f11f43974\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") "
Nov 04 02:12:42.611787 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.611724    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-credential-socket\") pod \"627ad113-e0fa-4637-9ac6-c71f11f43974\" (UID: \"627ad113-e0fa-4637-9ac6-c71f11f43974\") "
Nov 04 02:12:42.611997 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.611846    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-workload-certs" pod "627ad113-e0fa-4637-9ac6-c71f11f43974" (UID: "627ad113-e0fa-4637-9ac6-c71f11f43974"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:12:42.611997 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.611927    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/627ad113-e0fa-4637-9ac6-c71f11f43974-istiod-ca-cert" pod "627ad113-e0fa-4637-9ac6-c71f11f43974" (UID: "627ad113-e0fa-4637-9ac6-c71f11f43974"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:12:42.612106 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.612076    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-credential-socket" pod "627ad113-e0fa-4637-9ac6-c71f11f43974" (UID: "627ad113-e0fa-4637-9ac6-c71f11f43974"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:12:42.612183 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.612125    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-workload-socket" pod "627ad113-e0fa-4637-9ac6-c71f11f43974" (UID: "627ad113-e0fa-4637-9ac6-c71f11f43974"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:12:42.612244 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.612189    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-data" pod "627ad113-e0fa-4637-9ac6-c71f11f43974" (UID: "627ad113-e0fa-4637-9ac6-c71f11f43974"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:12:42.612372 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.612347    4865 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-workload-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.612446 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.612375    4865 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-workload-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.612446 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.612391    4865 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-data\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.612446 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.612406    4865 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-credential-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.612446 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.612420    4865 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/627ad113-e0fa-4637-9ac6-c71f11f43974-istiod-ca-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.613715 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.613688    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-token" pod "627ad113-e0fa-4637-9ac6-c71f11f43974" (UID: "627ad113-e0fa-4637-9ac6-c71f11f43974"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:12:42.613838 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.613806    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-envoy" pod "627ad113-e0fa-4637-9ac6-c71f11f43974" (UID: "627ad113-e0fa-4637-9ac6-c71f11f43974"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:12:42.613884 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.613875    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-podinfo" pod "627ad113-e0fa-4637-9ac6-c71f11f43974" (UID: "627ad113-e0fa-4637-9ac6-c71f11f43974"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:12:42.614057 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.614040    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/627ad113-e0fa-4637-9ac6-c71f11f43974-kube-api-access-snts4" pod "627ad113-e0fa-4637-9ac6-c71f11f43974" (UID: "627ad113-e0fa-4637-9ac6-c71f11f43974"). InnerVolumeSpecName "kube-api-access-snts4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:12:42.713684 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.713651    4865 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.713684 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.713682    4865 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-podinfo\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.713882 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.713695    4865 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/627ad113-e0fa-4637-9ac6-c71f11f43974-istio-envoy\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.713882 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.713708    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snts4\" (UniqueName: \"kubernetes.io/projected/627ad113-e0fa-4637-9ac6-c71f11f43974-kube-api-access-snts4\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.838712 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.838627    4865 generic.go:356] "Generic (PLEG): container finished" podID="627ad113-e0fa-4637-9ac6-c71f11f43974" containerID="2cbb541a50af951c6e3adda5cb62e37e78c7c5016d09a9986bff65574c2e204d" exitCode=0
Nov 04 02:12:42.838712 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.838678    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"
Nov 04 02:12:42.838887 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.838715    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w" event={"ID":"627ad113-e0fa-4637-9ac6-c71f11f43974","Type":"ContainerDied","Data":"2cbb541a50af951c6e3adda5cb62e37e78c7c5016d09a9986bff65574c2e204d"}
Nov 04 02:12:42.838887 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.838775    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w" event={"ID":"627ad113-e0fa-4637-9ac6-c71f11f43974","Type":"ContainerDied","Data":"7e7942441c5c51e6f48a3f3f33eb273b7da1ec18fcf656ea2a84fcde6b54dfd0"}
Nov 04 02:12:42.838887 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.838797    4865 scope.go:122] "RemoveContainer" containerID="2cbb541a50af951c6e3adda5cb62e37e78c7c5016d09a9986bff65574c2e204d"
Nov 04 02:12:42.848965 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.848949    4865 scope.go:122] "RemoveContainer" containerID="2cbb541a50af951c6e3adda5cb62e37e78c7c5016d09a9986bff65574c2e204d"
Nov 04 02:12:42.849259 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:12:42.849236    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cbb541a50af951c6e3adda5cb62e37e78c7c5016d09a9986bff65574c2e204d\": container with ID starting with 2cbb541a50af951c6e3adda5cb62e37e78c7c5016d09a9986bff65574c2e204d not found: ID does not exist" containerID="2cbb541a50af951c6e3adda5cb62e37e78c7c5016d09a9986bff65574c2e204d"
Nov 04 02:12:42.849365 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:42.849265    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cbb541a50af951c6e3adda5cb62e37e78c7c5016d09a9986bff65574c2e204d"} err="failed to get container status \"2cbb541a50af951c6e3adda5cb62e37e78c7c5016d09a9986bff65574c2e204d\": rpc error: code = NotFound desc = could not find container \"2cbb541a50af951c6e3adda5cb62e37e78c7c5016d09a9986bff65574c2e204d\": container with ID starting with 2cbb541a50af951c6e3adda5cb62e37e78c7c5016d09a9986bff65574c2e204d not found: ID does not exist"
Nov 04 02:12:43.007330 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:43.007291    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"]
Nov 04 02:12:43.014218 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:43.014190    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w"]
Nov 04 02:12:44.895116 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:44.895038    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="627ad113-e0fa-4637-9ac6-c71f11f43974" path="/var/lib/kubelet/pods/627ad113-e0fa-4637-9ac6-c71f11f43974/volumes"
Nov 04 02:12:58.403844 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:12:58.403808    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:12:58.911829 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:58.911799    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"]
Nov 04 02:12:58.912252 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:58.912238    4865 cpu_manager.go:414] "containerMap: removing container" podUID="627ad113-e0fa-4637-9ac6-c71f11f43974" containerName="istio-proxy"
Nov 04 02:12:58.912332 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:58.912254    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="627ad113-e0fa-4637-9ac6-c71f11f43974" containerName="istio-proxy"
Nov 04 02:12:58.912393 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:58.912383    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="627ad113-e0fa-4637-9ac6-c71f11f43974" containerName="istio-proxy"
Nov 04 02:12:58.915610 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:58.915591    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:58.976892 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:58.976855    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-model-cache\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:58.977071 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:58.976917    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-tls-certs\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:58.977134 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:58.977075    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs8jb\" (UniqueName: \"kubernetes.io/projected/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-kube-api-access-bs8jb\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:58.977242 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:58.977219    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-home\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:58.977319 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:58.977268    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-dshm\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:58.977375 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:58.977333    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-tmp-dir\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:59.022020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.021990    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-24bea9\"/\"distinct-w0-24bea9-kserve-self-signed-certs\""
Nov 04 02:12:59.024260 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.024237    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"]
Nov 04 02:12:59.078796 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.078758    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-tmp-dir\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:59.078985 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.078843    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-model-cache\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:59.078985 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.078876    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-tls-certs\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:59.078985 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.078918    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs8jb\" (UniqueName: \"kubernetes.io/projected/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-kube-api-access-bs8jb\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:59.079169 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.079002    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-home\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:59.079169 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.079037    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-dshm\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:59.079318 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.079207    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-model-cache\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:59.079318 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.079235    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-tmp-dir\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:59.079446 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.079421    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-home\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:59.081618 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.081590    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-dshm\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:59.081728 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.081615    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-tls-certs\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:59.141601 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.141569    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs8jb\" (UniqueName: \"kubernetes.io/projected/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-kube-api-access-bs8jb\") pod \"distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:59.231058 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.231026    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:12:59.610186 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.610160    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"]
Nov 04 02:12:59.611505 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:12:59.611478    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ab402a2_4ffb_4a93_b02d_5b8d1bda18f1.slice/crio-bc192d8070531b72274050a12e5a15a036a874b47d6f6dbe1b7df7146cae19e1 WatchSource:0}: Error finding container bc192d8070531b72274050a12e5a15a036a874b47d6f6dbe1b7df7146cae19e1: Status 404 returned error can't find the container with id bc192d8070531b72274050a12e5a15a036a874b47d6f6dbe1b7df7146cae19e1
Nov 04 02:12:59.926718 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.926616    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx" event={"ID":"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1","Type":"ContainerStarted","Data":"6d74aa9f1d49564a934082f6a44c6c2435ff923e3bd924d885f5dc5331684e70"}
Nov 04 02:12:59.926718 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:12:59.926660    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx" event={"ID":"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1","Type":"ContainerStarted","Data":"bc192d8070531b72274050a12e5a15a036a874b47d6f6dbe1b7df7146cae19e1"}
Nov 04 02:13:00.046776 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:00.046721    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx" podStartSLOduration=2.046705903 podStartE2EDuration="2.046705903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:12:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:13:00.044874663 +0000 UTC m=+1043.271661330" watchObservedRunningTime="2026-11-04 02:13:00.046705903 +0000 UTC m=+1043.273492573"
Nov 04 02:13:00.231651 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:00.231608    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:13:06.695923 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:06.695879    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:13:09.231374 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:09.231338    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:13:09.243783 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:09.243759    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:13:09.992775 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:09.992741    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:13:28.403228 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:13:28.403191    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:13:43.740788 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.740750    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"]
Nov 04 02:13:43.745706 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.745688    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.792161 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.792116    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-model-cache\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.792361 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.792229    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-dshm\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.792361 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.792255    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-tmp-dir\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.792361 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.792325    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-tls-certs\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.792546 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.792369    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bpcm\" (UniqueName: \"kubernetes.io/projected/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-kube-api-access-9bpcm\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.792546 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.792409    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-home\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.812863 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.812815    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-90ad0f\"/\"distinct-w1-90ad0f-kserve-self-signed-certs\""
Nov 04 02:13:43.850661 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.850633    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"]
Nov 04 02:13:43.893397 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.893364    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-model-cache\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.893560 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.893411    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-dshm\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.893560 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.893430    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-tmp-dir\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.893560 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.893465    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-tls-certs\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.893560 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.893522    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bpcm\" (UniqueName: \"kubernetes.io/projected/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-kube-api-access-9bpcm\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.893560 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.893548    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-home\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.893870 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.893845    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-model-cache\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.893923 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.893872    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-tmp-dir\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.893923 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.893912    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-home\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.895767 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.895745    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-dshm\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.896006 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.895988    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-tls-certs\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:43.955749 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:43.955718    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bpcm\" (UniqueName: \"kubernetes.io/projected/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-kube-api-access-9bpcm\") pod \"distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:44.057481 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:44.057399    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:44.433156 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:44.433133    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"]
Nov 04 02:13:44.434896 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:13:44.434871    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc04d1bc_8837_4ca4_b6fe_0f292a32d3c5.slice/crio-158271a3e30af6138d65d83b91d645ca6cbdbe72f8535ee52a0c79abd20a49e3 WatchSource:0}: Error finding container 158271a3e30af6138d65d83b91d645ca6cbdbe72f8535ee52a0c79abd20a49e3: Status 404 returned error can't find the container with id 158271a3e30af6138d65d83b91d645ca6cbdbe72f8535ee52a0c79abd20a49e3
Nov 04 02:13:45.170103 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:45.170060    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl" event={"ID":"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5","Type":"ContainerStarted","Data":"e1c0aa15d604bcf00bb651c1db3cf63b705184ce6536367912981f0cc63ca7cc"}
Nov 04 02:13:45.170103 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:45.170107    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl" event={"ID":"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5","Type":"ContainerStarted","Data":"158271a3e30af6138d65d83b91d645ca6cbdbe72f8535ee52a0c79abd20a49e3"}
Nov 04 02:13:45.314695 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:45.314641    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl" podStartSLOduration=2.314627312 podStartE2EDuration="2.314627312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:13:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:13:45.312894959 +0000 UTC m=+1088.539681626" watchObservedRunningTime="2026-11-04 02:13:45.314627312 +0000 UTC m=+1088.541413981"
Nov 04 02:13:46.058626 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.058594    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:46.172174 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.172137    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"]
Nov 04 02:13:46.176250 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.176226    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.218115 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.218062    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-credential-socket\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.218343 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.218210    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btn4b\" (UniqueName: \"kubernetes.io/projected/aeaab295-b237-407a-8b12-24da28c578c0-kube-api-access-btn4b\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.218343 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.218294    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-workload-certs\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.218478 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.218356    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-istio-data\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.218478 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.218416    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aeaab295-b237-407a-8b12-24da28c578c0-istio-token\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.218478 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.218452    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aeaab295-b237-407a-8b12-24da28c578c0-istiod-ca-cert\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.218647 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.218481    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aeaab295-b237-407a-8b12-24da28c578c0-istio-podinfo\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.218647 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.218510    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-workload-socket\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.218647 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.218563    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-istio-envoy\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.261478 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.261445    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-a8ef09-openshift-default-dockercfg-4qnv7\""
Nov 04 02:13:46.313775 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.313691    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"]
Nov 04 02:13:46.319477 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.319447    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btn4b\" (UniqueName: \"kubernetes.io/projected/aeaab295-b237-407a-8b12-24da28c578c0-kube-api-access-btn4b\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.319717 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.319695    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-workload-certs\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.319851 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.319837    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-istio-data\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.319973 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.319960    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aeaab295-b237-407a-8b12-24da28c578c0-istio-token\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.320075 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.320063    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aeaab295-b237-407a-8b12-24da28c578c0-istiod-ca-cert\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.320178 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.320163    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aeaab295-b237-407a-8b12-24da28c578c0-istio-podinfo\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.320277 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.320261    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-workload-socket\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.320430 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.320415    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-istio-envoy\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.320536 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.320523    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-credential-socket\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.321187 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.321161    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-credential-socket\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.321927 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.321901    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-workload-certs\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.322337 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.322316    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-istio-data\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.343305 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.325400    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-workload-socket\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.343305 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.336195    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aeaab295-b237-407a-8b12-24da28c578c0-istio-podinfo\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.344028 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.343998    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aeaab295-b237-407a-8b12-24da28c578c0-istiod-ca-cert\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.345402 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.345254    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-istio-envoy\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.383897 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.383861    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btn4b\" (UniqueName: \"kubernetes.io/projected/aeaab295-b237-407a-8b12-24da28c578c0-kube-api-access-btn4b\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.384870 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.384845    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aeaab295-b237-407a-8b12-24da28c578c0-istio-token\") pod \"e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") " pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.491245 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.491206    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:46.925502 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.925476    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"]
Nov 04 02:13:46.926722 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:13:46.926701    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeaab295_b237_407a_8b12_24da28c578c0.slice/crio-03d70d56573b81a797d25eaf034306b3d9f9a4f86b7f4525bcf4cd311a344d11 WatchSource:0}: Error finding container 03d70d56573b81a797d25eaf034306b3d9f9a4f86b7f4525bcf4cd311a344d11: Status 404 returned error can't find the container with id 03d70d56573b81a797d25eaf034306b3d9f9a4f86b7f4525bcf4cd311a344d11
Nov 04 02:13:46.928880 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.928851    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:13:46.928944 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.928911    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:13:46.928944 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:46.928941    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:13:47.184023 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:47.183908    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp" event={"ID":"aeaab295-b237-407a-8b12-24da28c578c0","Type":"ContainerStarted","Data":"435daf6c70478daf0497976300c458eadc3e49e5757d89ae535cb5ba5ff0cfc8"}
Nov 04 02:13:47.184023 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:47.183985    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp" event={"ID":"aeaab295-b237-407a-8b12-24da28c578c0","Type":"ContainerStarted","Data":"03d70d56573b81a797d25eaf034306b3d9f9a4f86b7f4525bcf4cd311a344d11"}
Nov 04 02:13:47.309974 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:47.309918    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp" podStartSLOduration=1.30990345 podStartE2EDuration="1.30990345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:13:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:13:47.307444981 +0000 UTC m=+1090.534231650" watchObservedRunningTime="2026-11-04 02:13:47.30990345 +0000 UTC m=+1090.536690117"
Nov 04 02:13:47.492182 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:47.492144    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:47.497681 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:47.497658    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:48.189989 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:48.189956    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:48.190960 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:48.190937    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:13:54.058142 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:54.058091    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:54.079182 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:54.079154    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:54.231558 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:54.231525    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:13:55.056684 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:13:55.056653    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:13:58.404154 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:13:58.404113    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:14:18.881833 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:18.881745    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"]
Nov 04 02:14:18.887026 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:18.887002    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:14:18.933929 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:18.933902    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da8fda5b-215f-4c95-9f7f-31146a1fd90d-maas-api-tls\") pod \"maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h\" (UID: \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:14:18.934059 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:18.933939    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da8fda5b-215f-4c95-9f7f-31146a1fd90d-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h\" (UID: \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:14:18.934059 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:18.934049    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq7ch\" (UniqueName: \"kubernetes.io/projected/da8fda5b-215f-4c95-9f7f-31146a1fd90d-kube-api-access-fq7ch\") pod \"maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h\" (UID: \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:14:18.934148 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:18.934137    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da8fda5b-215f-4c95-9f7f-31146a1fd90d-metrics-certs\") pod \"maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h\" (UID: \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:14:18.994565 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:18.994531    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-a8ef09-serving-cert\""
Nov 04 02:14:18.997026 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:18.997002    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"]
Nov 04 02:14:19.034918 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:19.034883    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da8fda5b-215f-4c95-9f7f-31146a1fd90d-metrics-certs\") pod \"maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h\" (UID: \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:14:19.035062 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:19.035013    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da8fda5b-215f-4c95-9f7f-31146a1fd90d-maas-api-tls\") pod \"maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h\" (UID: \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:14:19.035062 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:19.035038    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da8fda5b-215f-4c95-9f7f-31146a1fd90d-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h\" (UID: \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:14:19.035178 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:19.035100    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq7ch\" (UniqueName: \"kubernetes.io/projected/da8fda5b-215f-4c95-9f7f-31146a1fd90d-kube-api-access-fq7ch\") pod \"maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h\" (UID: \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:14:19.035178 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:14:19.035156    4865 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-serving-cert: secret "maas-api-e2e-extmod-a-a8ef09-serving-cert" not found
Nov 04 02:14:19.035652 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:14:19.035631    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da8fda5b-215f-4c95-9f7f-31146a1fd90d-maas-api-tls podName:da8fda5b-215f-4c95-9f7f-31146a1fd90d nodeName:}" failed. No retries permitted until 2026-11-04 02:14:19.535604526 +0000 UTC m=+1122.762391173 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/da8fda5b-215f-4c95-9f7f-31146a1fd90d-maas-api-tls") pod "maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h" (UID: "da8fda5b-215f-4c95-9f7f-31146a1fd90d") : secret "maas-api-e2e-extmod-a-a8ef09-serving-cert" not found
Nov 04 02:14:19.036015 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:19.035986    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da8fda5b-215f-4c95-9f7f-31146a1fd90d-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h\" (UID: \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:14:19.037820 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:19.037798    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da8fda5b-215f-4c95-9f7f-31146a1fd90d-metrics-certs\") pod \"maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h\" (UID: \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:14:19.099357 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:19.099333    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq7ch\" (UniqueName: \"kubernetes.io/projected/da8fda5b-215f-4c95-9f7f-31146a1fd90d-kube-api-access-fq7ch\") pod \"maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h\" (UID: \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:14:19.541008 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:19.540976    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da8fda5b-215f-4c95-9f7f-31146a1fd90d-maas-api-tls\") pod \"maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h\" (UID: \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:14:19.543478 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:19.543457    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da8fda5b-215f-4c95-9f7f-31146a1fd90d-maas-api-tls\") pod \"maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h\" (UID: \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:14:19.801959 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:19.801878    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:14:20.186177 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:20.186146    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"]
Nov 04 02:14:20.186893 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:14:20.186865    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda8fda5b_215f_4c95_9f7f_31146a1fd90d.slice/crio-7b42b19c66cbdde98a268533ddf49995b1b2f8bbc5ec900580d1519ae5f61e04 WatchSource:0}: Error finding container 7b42b19c66cbdde98a268533ddf49995b1b2f8bbc5ec900580d1519ae5f61e04: Status 404 returned error can't find the container with id 7b42b19c66cbdde98a268533ddf49995b1b2f8bbc5ec900580d1519ae5f61e04
Nov 04 02:14:20.188244 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:20.188223    4865 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:14:20.378957 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:20.378920    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h" event={"ID":"da8fda5b-215f-4c95-9f7f-31146a1fd90d","Type":"ContainerStarted","Data":"7b42b19c66cbdde98a268533ddf49995b1b2f8bbc5ec900580d1519ae5f61e04"}
Nov 04 02:14:21.391907 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:21.391869    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h" event={"ID":"da8fda5b-215f-4c95-9f7f-31146a1fd90d","Type":"ContainerStarted","Data":"9f00d25904a619f5195d9cf186e58a0cc5b1d01a791d777563fa266b6695e09a"}
Nov 04 02:14:21.392374 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:21.391982    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:14:21.511179 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:21.511118    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h" podStartSLOduration=2.554098097 podStartE2EDuration="3.511099151s" totalImagesPullingTime="957.001054ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:14:21.510603713 +0000 UTC m=+1124.737390382" watchObservedRunningTime="2026-11-04 02:14:21.511099151 +0000 UTC m=+1124.737885824"
Nov 04 02:14:24.984452 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:24.984414    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:14:27.402883 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:27.402798    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:14:28.403737 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:14:28.403700    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:14:37.318383 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:37.318348    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"]
Nov 04 02:14:37.318884 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:37.318552    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v" podUID="52289639-1d6e-4d31-b2da-49277feea25d" containerName="istio-proxy" containerID="cri-o://277219f201d104d648af53662f0c6662c39e13db499d6ff180003b9b4d0c657d" gracePeriod=30
Nov 04 02:14:42.531061 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.530879    4865 generic.go:356] "Generic (PLEG): container finished" podID="52289639-1d6e-4d31-b2da-49277feea25d" containerID="277219f201d104d648af53662f0c6662c39e13db499d6ff180003b9b4d0c657d" exitCode=0
Nov 04 02:14:42.531061 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.530991    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v" event={"ID":"52289639-1d6e-4d31-b2da-49277feea25d","Type":"ContainerDied","Data":"277219f201d104d648af53662f0c6662c39e13db499d6ff180003b9b4d0c657d"}
Nov 04 02:14:42.628613 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.628579    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:14:42.633697 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.633673    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/52289639-1d6e-4d31-b2da-49277feea25d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52289639-1d6e-4d31-b2da-49277feea25d-istio-podinfo\") pod \"52289639-1d6e-4d31-b2da-49277feea25d\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") "
Nov 04 02:14:42.633800 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.633705    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52289639-1d6e-4d31-b2da-49277feea25d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52289639-1d6e-4d31-b2da-49277feea25d-istiod-ca-cert\") pod \"52289639-1d6e-4d31-b2da-49277feea25d\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") "
Nov 04 02:14:42.633800 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.633740    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-workload-socket\") pod \"52289639-1d6e-4d31-b2da-49277feea25d\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") "
Nov 04 02:14:42.633800 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.633767    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-workload-certs\") pod \"52289639-1d6e-4d31-b2da-49277feea25d\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") "
Nov 04 02:14:42.633945 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.633819    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52289639-1d6e-4d31-b2da-49277feea25d-istio-token\" (UniqueName: \"kubernetes.io/projected/52289639-1d6e-4d31-b2da-49277feea25d-istio-token\") pod \"52289639-1d6e-4d31-b2da-49277feea25d\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") "
Nov 04 02:14:42.633945 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.633875    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-istio-envoy\") pod \"52289639-1d6e-4d31-b2da-49277feea25d\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") "
Nov 04 02:14:42.634049 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.633977    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-credential-socket\") pod \"52289639-1d6e-4d31-b2da-49277feea25d\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") "
Nov 04 02:14:42.634103 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.634084    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-istio-data\") pod \"52289639-1d6e-4d31-b2da-49277feea25d\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") "
Nov 04 02:14:42.634159 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.634126    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52289639-1d6e-4d31-b2da-49277feea25d-kube-api-access-jzvr8\" (UniqueName: \"kubernetes.io/projected/52289639-1d6e-4d31-b2da-49277feea25d-kube-api-access-jzvr8\") pod \"52289639-1d6e-4d31-b2da-49277feea25d\" (UID: \"52289639-1d6e-4d31-b2da-49277feea25d\") "
Nov 04 02:14:42.634214 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.634145    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-workload-socket" pod "52289639-1d6e-4d31-b2da-49277feea25d" (UID: "52289639-1d6e-4d31-b2da-49277feea25d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:14:42.634214 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.634163    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-workload-certs" pod "52289639-1d6e-4d31-b2da-49277feea25d" (UID: "52289639-1d6e-4d31-b2da-49277feea25d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:14:42.634214 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.634184    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52289639-1d6e-4d31-b2da-49277feea25d-istiod-ca-cert" pod "52289639-1d6e-4d31-b2da-49277feea25d" (UID: "52289639-1d6e-4d31-b2da-49277feea25d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:14:42.634450 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.634403    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-credential-socket" pod "52289639-1d6e-4d31-b2da-49277feea25d" (UID: "52289639-1d6e-4d31-b2da-49277feea25d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:14:42.634450 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.634419    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-istio-data" pod "52289639-1d6e-4d31-b2da-49277feea25d" (UID: "52289639-1d6e-4d31-b2da-49277feea25d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:14:42.634649 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.634603    4865 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-credential-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:14:42.634649 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.634622    4865 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-istio-data\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:14:42.634649 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.634636    4865 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52289639-1d6e-4d31-b2da-49277feea25d-istiod-ca-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:14:42.634817 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.634649    4865 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-workload-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:14:42.634817 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.634662    4865 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-workload-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:14:42.636089 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.636062    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/52289639-1d6e-4d31-b2da-49277feea25d-istio-podinfo" pod "52289639-1d6e-4d31-b2da-49277feea25d" (UID: "52289639-1d6e-4d31-b2da-49277feea25d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:14:42.636584 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.636553    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52289639-1d6e-4d31-b2da-49277feea25d-kube-api-access-jzvr8" pod "52289639-1d6e-4d31-b2da-49277feea25d" (UID: "52289639-1d6e-4d31-b2da-49277feea25d"). InnerVolumeSpecName "kube-api-access-jzvr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:14:42.637038 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.637012    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52289639-1d6e-4d31-b2da-49277feea25d-istio-token" pod "52289639-1d6e-4d31-b2da-49277feea25d" (UID: "52289639-1d6e-4d31-b2da-49277feea25d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:14:42.637363 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.637341    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-istio-envoy" pod "52289639-1d6e-4d31-b2da-49277feea25d" (UID: "52289639-1d6e-4d31-b2da-49277feea25d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:14:42.736196 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.736170    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzvr8\" (UniqueName: \"kubernetes.io/projected/52289639-1d6e-4d31-b2da-49277feea25d-kube-api-access-jzvr8\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:14:42.736196 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.736195    4865 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52289639-1d6e-4d31-b2da-49277feea25d-istio-podinfo\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:14:42.736397 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.736205    4865 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52289639-1d6e-4d31-b2da-49277feea25d-istio-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:14:42.736397 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:42.736214    4865 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52289639-1d6e-4d31-b2da-49277feea25d-istio-envoy\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:14:43.573641 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:43.573613    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"
Nov 04 02:14:43.574177 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:43.573666    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v" event={"ID":"52289639-1d6e-4d31-b2da-49277feea25d","Type":"ContainerDied","Data":"595757bffb4fa69f6b056b35688a5f52ca6f98728fc88c9ab2396124361b47ca"}
Nov 04 02:14:43.574177 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:43.573743    4865 scope.go:122] "RemoveContainer" containerID="277219f201d104d648af53662f0c6662c39e13db499d6ff180003b9b4d0c657d"
Nov 04 02:14:43.761680 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:43.761642    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"]
Nov 04 02:14:43.794944 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:43.794909    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v"]
Nov 04 02:14:44.896103 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:44.896070    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="52289639-1d6e-4d31-b2da-49277feea25d" path="/var/lib/kubelet/pods/52289639-1d6e-4d31-b2da-49277feea25d/volumes"
Nov 04 02:14:51.621361 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:51.621331    4865 generic.go:356] "Generic (PLEG): container finished" podID="aeaab295-b237-407a-8b12-24da28c578c0" containerID="435daf6c70478daf0497976300c458eadc3e49e5757d89ae535cb5ba5ff0cfc8" exitCode=0
Nov 04 02:14:51.621798 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:51.621391    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp" event={"ID":"aeaab295-b237-407a-8b12-24da28c578c0","Type":"ContainerDied","Data":"435daf6c70478daf0497976300c458eadc3e49e5757d89ae535cb5ba5ff0cfc8"}
Nov 04 02:14:51.621798 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:51.621748    4865 scope.go:122] "RemoveContainer" containerID="435daf6c70478daf0497976300c458eadc3e49e5757d89ae535cb5ba5ff0cfc8"
Nov 04 02:14:51.623024 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:51.622995    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:14:51.623078 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:51.623064    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:14:51.623113 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:51.623099    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:14:52.628762 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:52.628724    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp" event={"ID":"aeaab295-b237-407a-8b12-24da28c578c0","Type":"ContainerStarted","Data":"cf011bed5f5fcf89f8b97e180d10aaeb4925b4372a096b4a9cc6a75a45129434"}
Nov 04 02:14:53.491882 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:53.491849    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:14:53.492086 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:53.492017    4865 patch_prober.go:29] interesting pod/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 04 02:14:53.492086 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:53.492070    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp" podUID="aeaab295-b237-407a-8b12-24da28c578c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 04 02:14:54.492109 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:54.492069    4865 patch_prober.go:29] interesting pod/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 04 02:14:54.492561 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:54.492137    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp" podUID="aeaab295-b237-407a-8b12-24da28c578c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 04 02:14:55.491764 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:55.491728    4865 patch_prober.go:29] interesting pod/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 04 02:14:55.491926 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:55.491798    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp" podUID="aeaab295-b237-407a-8b12-24da28c578c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 04 02:14:56.492005 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:56.491973    4865 patch_prober.go:29] interesting pod/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 04 02:14:56.492432 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:56.492037    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp" podUID="aeaab295-b237-407a-8b12-24da28c578c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 04 02:14:57.492715 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:57.492679    4865 patch_prober.go:29] interesting pod/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 04 02:14:57.493092 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:57.492741    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp" podUID="aeaab295-b237-407a-8b12-24da28c578c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 04 02:14:58.402933 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:14:58.402893    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:14:58.442337 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.440706    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"]
Nov 04 02:14:58.442337 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.441953    4865 cpu_manager.go:414] "containerMap: removing container" podUID="52289639-1d6e-4d31-b2da-49277feea25d" containerName="istio-proxy"
Nov 04 02:14:58.442337 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.441974    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52289639-1d6e-4d31-b2da-49277feea25d" containerName="istio-proxy"
Nov 04 02:14:58.442337 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.442220    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="52289639-1d6e-4d31-b2da-49277feea25d" containerName="istio-proxy"
Nov 04 02:14:58.445437 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.445411    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.497200 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.497167    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:14:58.497638 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.497618    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:14:58.498401 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.498380    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:14:58.517720 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.517687    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"]
Nov 04 02:14:58.521930 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.521903    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-dshm\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.522054 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.521943    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-model-cache\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.522054 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.521999    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcp4n\" (UniqueName: \"kubernetes.io/projected/6f585314-0bb4-48b5-9b4d-80a4e244089c-kube-api-access-jcp4n\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.522211 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.522170    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-home\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.522317 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.522277    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-tmp-dir\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.522456 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.522438    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f585314-0bb4-48b5-9b4d-80a4e244089c-tls-certs\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.550602 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.550578    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-90ad0f\"/\"distinct-2-w1-90ad0f-kserve-self-signed-certs\""
Nov 04 02:14:58.623190 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.623152    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-tmp-dir\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.623398 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.623261    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f585314-0bb4-48b5-9b4d-80a4e244089c-tls-certs\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.623398 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.623358    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-dshm\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.623398 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.623382    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-model-cache\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.623536 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.623435    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcp4n\" (UniqueName: \"kubernetes.io/projected/6f585314-0bb4-48b5-9b4d-80a4e244089c-kube-api-access-jcp4n\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.623536 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.623475    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-home\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.623687 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.623664    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-tmp-dir\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.623777 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.623756    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-model-cache\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.623858 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.623838    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-home\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.625628 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.625610    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-dshm\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.625793 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.625767    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f585314-0bb4-48b5-9b4d-80a4e244089c-tls-certs\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.686237 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.686164    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcp4n\" (UniqueName: \"kubernetes.io/projected/6f585314-0bb4-48b5-9b4d-80a4e244089c-kube-api-access-jcp4n\") pod \"distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") " pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:58.758173 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:58.758138    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:59.181134 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:59.181105    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"]
Nov 04 02:14:59.181254 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:14:59.181222    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f585314_0bb4_48b5_9b4d_80a4e244089c.slice/crio-1663c9ca4050b47656a884d77201f33bcc5c03d1a7d10bfd059c25f4e54f0410 WatchSource:0}: Error finding container 1663c9ca4050b47656a884d77201f33bcc5c03d1a7d10bfd059c25f4e54f0410: Status 404 returned error can't find the container with id 1663c9ca4050b47656a884d77201f33bcc5c03d1a7d10bfd059c25f4e54f0410
Nov 04 02:14:59.581970 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:59.581933    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:14:59.693765 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:59.693731    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq" event={"ID":"6f585314-0bb4-48b5-9b4d-80a4e244089c","Type":"ContainerStarted","Data":"7833b88efd58ca5cdfeaf30bc1eaf430db6da8c107bd4a7165317f17e7a860f1"}
Nov 04 02:14:59.693940 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:59.693781    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq" event={"ID":"6f585314-0bb4-48b5-9b4d-80a4e244089c","Type":"ContainerStarted","Data":"1663c9ca4050b47656a884d77201f33bcc5c03d1a7d10bfd059c25f4e54f0410"}
Nov 04 02:14:59.759096 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:59.759066    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:14:59.814395 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:14:59.814349    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq" podStartSLOduration=1.814335907 podStartE2EDuration="1.814335907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:14:59.812400036 +0000 UTC m=+1163.039186703" watchObservedRunningTime="2026-11-04 02:14:59.814335907 +0000 UTC m=+1163.041122576"
Nov 04 02:15:00.200482 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:00.200445    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld"]
Nov 04 02:15:00.203457 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:00.203441    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld"
Nov 04 02:15:00.246585 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:00.246552    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a76164fe-da41-44fa-b5c7-31cceee69d17-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld\" (UID: \"a76164fe-da41-44fa-b5c7-31cceee69d17\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld"
Nov 04 02:15:00.246754 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:00.246691    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwdzs\" (UniqueName: \"kubernetes.io/projected/a76164fe-da41-44fa-b5c7-31cceee69d17-kube-api-access-wwdzs\") pod \"maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld\" (UID: \"a76164fe-da41-44fa-b5c7-31cceee69d17\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld"
Nov 04 02:15:00.280186 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:00.280147    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld"]
Nov 04 02:15:00.334391 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:00.334356    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:15:00.347827 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:00.347795    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a76164fe-da41-44fa-b5c7-31cceee69d17-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld\" (UID: \"a76164fe-da41-44fa-b5c7-31cceee69d17\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld"
Nov 04 02:15:00.347985 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:00.347906    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwdzs\" (UniqueName: \"kubernetes.io/projected/a76164fe-da41-44fa-b5c7-31cceee69d17-kube-api-access-wwdzs\") pod \"maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld\" (UID: \"a76164fe-da41-44fa-b5c7-31cceee69d17\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld"
Nov 04 02:15:00.348454 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:00.348433    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a76164fe-da41-44fa-b5c7-31cceee69d17-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld\" (UID: \"a76164fe-da41-44fa-b5c7-31cceee69d17\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld"
Nov 04 02:15:00.409734 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:00.409700    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwdzs\" (UniqueName: \"kubernetes.io/projected/a76164fe-da41-44fa-b5c7-31cceee69d17-kube-api-access-wwdzs\") pod \"maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld\" (UID: \"a76164fe-da41-44fa-b5c7-31cceee69d17\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld"
Nov 04 02:15:00.517378 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:00.517249    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld"
Nov 04 02:15:00.897383 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:00.897359    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld"]
Nov 04 02:15:00.899124 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:15:00.899099    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda76164fe_da41_44fa_b5c7_31cceee69d17.slice/crio-7607952220c7746fb11b0855f83a0eaeb3e52c335d92dfca8999d40afbdd6556 WatchSource:0}: Error finding container 7607952220c7746fb11b0855f83a0eaeb3e52c335d92dfca8999d40afbdd6556: Status 404 returned error can't find the container with id 7607952220c7746fb11b0855f83a0eaeb3e52c335d92dfca8999d40afbdd6556
Nov 04 02:15:01.705470 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:01.705433    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld" event={"ID":"a76164fe-da41-44fa-b5c7-31cceee69d17","Type":"ContainerStarted","Data":"7607952220c7746fb11b0855f83a0eaeb3e52c335d92dfca8999d40afbdd6556"}
Nov 04 02:15:03.665211 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.665181    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"]
Nov 04 02:15:03.668212 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.668184    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.770488 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.770453    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"]
Nov 04 02:15:03.788756 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.788354    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-token\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.789311 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.789007    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-workload-socket\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.789311 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.789089    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istiod-ca-cert\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.789311 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.789133    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-data\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.789311 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.789158    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-credential-socket\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.789311 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.789213    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-envoy\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.789311 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.789271    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-workload-certs\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.789656 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.789532    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49zl2\" (UniqueName: \"kubernetes.io/projected/e211ff43-fa6b-4912-9c5b-bc5cec322b03-kube-api-access-49zl2\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.789656 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.789601    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-podinfo\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.827176 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.824869    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-055a14-openshift-default-dockercfg-27mkm\""
Nov 04 02:15:03.890657 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.890616    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-token\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.890846 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.890676    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-workload-socket\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.890846 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.890720    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istiod-ca-cert\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.890846 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.890770    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-data\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.890846 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.890800    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-credential-socket\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.891029 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.890857    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-envoy\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.891029 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.890900    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-workload-certs\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.891029 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.890977    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49zl2\" (UniqueName: \"kubernetes.io/projected/e211ff43-fa6b-4912-9c5b-bc5cec322b03-kube-api-access-49zl2\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.891135 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.891038    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-podinfo\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.892078 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.892051    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-data\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.892202 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.892077    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-workload-socket\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.892756 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.892704    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-workload-certs\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.892821 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.892772    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istiod-ca-cert\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.893064 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.893044    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-credential-socket\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.894246 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.894221    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-podinfo\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.894400 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.894379    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-envoy\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.956316 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.956249    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-token\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.956316 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.956250    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49zl2\" (UniqueName: \"kubernetes.io/projected/e211ff43-fa6b-4912-9c5b-bc5cec322b03-kube-api-access-49zl2\") pod \"e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") " pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.983776 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.983748    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:03.990120 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.990094    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"]
Nov 04 02:15:03.993160 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:03.993140    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.056617 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.056575    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"]
Nov 04 02:15:04.068956 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.068917    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-0508d1-openshift-default-dockercfg-sm7sq\""
Nov 04 02:15:04.093243 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.093205    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91d9868b-f91c-424b-8671-f2973afdfbfa-istiod-ca-cert\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.093421 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.093366    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-credential-socket\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.093421 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.093407    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-workload-certs\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.093509 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.093431    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-podinfo\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.093509 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.093458    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-data\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.093509 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.093487    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqdcg\" (UniqueName: \"kubernetes.io/projected/91d9868b-f91c-424b-8671-f2973afdfbfa-kube-api-access-cqdcg\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.093623 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.093513    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-token\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.093623 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.093580    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-envoy\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.093711 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.093651    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-workload-socket\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.194463 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.194425    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-credential-socket\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.194629 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.194472    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-workload-certs\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.194629 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.194505    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-podinfo\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.194629 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.194534    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-data\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.194629 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.194564    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqdcg\" (UniqueName: \"kubernetes.io/projected/91d9868b-f91c-424b-8671-f2973afdfbfa-kube-api-access-cqdcg\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.194629 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.194593    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-token\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.194817 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.194662    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-envoy\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.194817 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.194733    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-workload-socket\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.194817 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.194808    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91d9868b-f91c-424b-8671-f2973afdfbfa-istiod-ca-cert\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.195113 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.194855    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-credential-socket\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.195531 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.195506    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-workload-certs\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.196054 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.196008    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-data\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.196464 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.196437    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-workload-socket\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.196464 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.196456    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91d9868b-f91c-424b-8671-f2973afdfbfa-istiod-ca-cert\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.198583 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.198562    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-envoy\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.198955 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.198934    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-podinfo\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.258381 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.257732    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-token\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.258381 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.258267    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqdcg\" (UniqueName: \"kubernetes.io/projected/91d9868b-f91c-424b-8671-f2973afdfbfa-kube-api-access-cqdcg\") pod \"e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") " pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.329429 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.329372    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:04.428959 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.428927    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"]
Nov 04 02:15:04.443058 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.443017    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:15:04.443168 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.443109    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:15:04.443168 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.443155    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:15:04.725913 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.725880    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"]
Nov 04 02:15:04.730219 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.730179    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:15:04.730346 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.730267    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:15:04.730393 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.730343    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:15:04.738748 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.737779    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l" event={"ID":"e211ff43-fa6b-4912-9c5b-bc5cec322b03","Type":"ContainerStarted","Data":"658d4407678b49127be650b8863da502047b56d33b7729eb7df4914af010ddfe"}
Nov 04 02:15:04.738748 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.737842    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l" event={"ID":"e211ff43-fa6b-4912-9c5b-bc5cec322b03","Type":"ContainerStarted","Data":"945aaad5c431dcf9f0410e74a0f8c6ddde2e0f944ee59ac65c10e6085ace232e"}
Nov 04 02:15:04.866579 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:04.865215    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l" podStartSLOduration=1.8652003609999999 podStartE2EDuration="1.865200361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:04.863818876 +0000 UTC m=+1168.090605543" watchObservedRunningTime="2026-11-04 02:15:04.865200361 +0000 UTC m=+1168.091987029"
Nov 04 02:15:05.006196 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:05.002130    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:05.013435 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:05.012236    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:05.745660 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:05.745626    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld" event={"ID":"a76164fe-da41-44fa-b5c7-31cceee69d17","Type":"ContainerStarted","Data":"380b412d84960f0a89d892ea7dc1ab11252d7dae4e43821d17e4a0103583d1c6"}
Nov 04 02:15:05.747476 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:05.747445    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql" event={"ID":"91d9868b-f91c-424b-8671-f2973afdfbfa","Type":"ContainerStarted","Data":"938e6fe08c4824de9f1c46784d8e7b6509d0d3e2f3052fdb91bc1f8f601c09c3"}
Nov 04 02:15:05.747609 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:05.747496    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql" event={"ID":"91d9868b-f91c-424b-8671-f2973afdfbfa","Type":"ContainerStarted","Data":"e43ee408cb22466d5fa7a3292c0fcc7d0855cbd95f253615b6539b683582ced7"}
Nov 04 02:15:05.747609 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:05.747592    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:05.748550 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:05.748534    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:15:05.864337 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:05.864270    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld" podStartSLOduration=1.580366881 podStartE2EDuration="5.86425645s" totalImagesPullingTime="4.283889569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:05.862604608 +0000 UTC m=+1169.089391277" watchObservedRunningTime="2026-11-04 02:15:05.86425645 +0000 UTC m=+1169.091043097"
Nov 04 02:15:05.958064 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:05.958026    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"]
Nov 04 02:15:05.962027 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:05.962001    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:05.986653 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:05.986607    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql" podStartSLOduration=2.986593537 podStartE2EDuration="2.986593537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:05.983966056 +0000 UTC m=+1169.210752725" watchObservedRunningTime="2026-11-04 02:15:05.986593537 +0000 UTC m=+1169.213380207"
Nov 04 02:15:06.017200 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.017113    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.017389 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.017238    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.017389 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.017314    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk2rb\" (UniqueName: \"kubernetes.io/projected/746c1e17-96d9-40fd-8cce-8aebfbc17666-kube-api-access-qk2rb\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.017481 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.017398    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.017481 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.017446    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.017563 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.017485    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/746c1e17-96d9-40fd-8cce-8aebfbc17666-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.017597 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.017562    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.017635 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.017594    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.017635 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.017613    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.022594 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.022567    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"]
Nov 04 02:15:06.143023 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.142981    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.143228 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.143048    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk2rb\" (UniqueName: \"kubernetes.io/projected/746c1e17-96d9-40fd-8cce-8aebfbc17666-kube-api-access-qk2rb\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.143228 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.143089    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.143228 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.143132    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.143228 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.143168    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/746c1e17-96d9-40fd-8cce-8aebfbc17666-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.143228 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.143217    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.143541 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.143269    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.143541 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.143334    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.143541 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.143404    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.144272 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.144243    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/746c1e17-96d9-40fd-8cce-8aebfbc17666-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.145605 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.144907    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.145605 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.145118    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.145805 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.145348    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.145805 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.145550    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.149862 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.149814    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.156235 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.156197    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.210490 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.210462    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.211967 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.211948    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk2rb\" (UniqueName: \"kubernetes.io/projected/746c1e17-96d9-40fd-8cce-8aebfbc17666-kube-api-access-qk2rb\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.276030 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.275934    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:06.331086 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.331002    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:06.339492 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.339468    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:06.711546 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.711508    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"]
Nov 04 02:15:06.713019 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:15:06.712988    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod746c1e17_96d9_40fd_8cce_8aebfbc17666.slice/crio-55681a229eac391eb14fd425fa0e398e23befec27cbd9f2faf7fca686a23bb40 WatchSource:0}: Error finding container 55681a229eac391eb14fd425fa0e398e23befec27cbd9f2faf7fca686a23bb40: Status 404 returned error can't find the container with id 55681a229eac391eb14fd425fa0e398e23befec27cbd9f2faf7fca686a23bb40
Nov 04 02:15:06.715503 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.715473    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:15:06.715566 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.715537    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:15:06.715602 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.715566    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:15:06.755924 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.755888    4865 generic.go:356] "Generic (PLEG): container finished" podID="a76164fe-da41-44fa-b5c7-31cceee69d17" containerID="380b412d84960f0a89d892ea7dc1ab11252d7dae4e43821d17e4a0103583d1c6" exitCode=60
Nov 04 02:15:06.756332 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.755967    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld" event={"ID":"a76164fe-da41-44fa-b5c7-31cceee69d17","Type":"ContainerDied","Data":"380b412d84960f0a89d892ea7dc1ab11252d7dae4e43821d17e4a0103583d1c6"}
Nov 04 02:15:06.756332 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.756199    4865 scope.go:122] "RemoveContainer" containerID="380b412d84960f0a89d892ea7dc1ab11252d7dae4e43821d17e4a0103583d1c6"
Nov 04 02:15:06.757514 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.757489    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2" event={"ID":"746c1e17-96d9-40fd-8cce-8aebfbc17666","Type":"ContainerStarted","Data":"55681a229eac391eb14fd425fa0e398e23befec27cbd9f2faf7fca686a23bb40"}
Nov 04 02:15:06.757908 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.757886    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:06.759622 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:06.759603    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:15:07.769157 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:07.769121    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2" event={"ID":"746c1e17-96d9-40fd-8cce-8aebfbc17666","Type":"ContainerStarted","Data":"a3f20a0df7b7913dbb4c9759dab1a8ae34d90512cfca64e2446dfbdac18875d7"}
Nov 04 02:15:07.771013 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:07.770990    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld" event={"ID":"a76164fe-da41-44fa-b5c7-31cceee69d17","Type":"ContainerStarted","Data":"92fdba7b77ad36ee25242602aeb3449a5865b028ea13e0c22909560961344eac"}
Nov 04 02:15:07.771237 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:07.771220    4865 scope.go:122] "RemoveContainer" containerID="92fdba7b77ad36ee25242602aeb3449a5865b028ea13e0c22909560961344eac"
Nov 04 02:15:07.771549 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:15:07.771526    4865 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld_odh-ai-gateway-infra(a76164fe-da41-44fa-b5c7-31cceee69d17)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld" podUID="a76164fe-da41-44fa-b5c7-31cceee69d17"
Nov 04 02:15:07.892871 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:07.892816    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2" podStartSLOduration=2.892803668 podStartE2EDuration="2.892803668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:07.892190347 +0000 UTC m=+1171.118977014" watchObservedRunningTime="2026-11-04 02:15:07.892803668 +0000 UTC m=+1171.119590335"
Nov 04 02:15:08.277138 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:08.277105    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:08.278689 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:08.278660    4865 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 02:15:08.278809 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:08.278740    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2" podUID="746c1e17-96d9-40fd-8cce-8aebfbc17666" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 02:15:08.758425 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:08.758385    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:15:08.773928 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:08.773887    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:15:08.780554 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:08.780524    4865 generic.go:356] "Generic (PLEG): container finished" podID="a76164fe-da41-44fa-b5c7-31cceee69d17" containerID="92fdba7b77ad36ee25242602aeb3449a5865b028ea13e0c22909560961344eac" exitCode=60
Nov 04 02:15:08.780718 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:08.780616    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld" event={"ID":"a76164fe-da41-44fa-b5c7-31cceee69d17","Type":"ContainerDied","Data":"92fdba7b77ad36ee25242602aeb3449a5865b028ea13e0c22909560961344eac"}
Nov 04 02:15:08.780718 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:08.780672    4865 scope.go:122] "RemoveContainer" containerID="380b412d84960f0a89d892ea7dc1ab11252d7dae4e43821d17e4a0103583d1c6"
Nov 04 02:15:08.780860 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:08.780841    4865 scope.go:122] "RemoveContainer" containerID="92fdba7b77ad36ee25242602aeb3449a5865b028ea13e0c22909560961344eac"
Nov 04 02:15:08.781102 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:15:08.781078    4865 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld_odh-ai-gateway-infra(a76164fe-da41-44fa-b5c7-31cceee69d17)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld" podUID="a76164fe-da41-44fa-b5c7-31cceee69d17"
Nov 04 02:15:08.797893 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:08.797869    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:15:09.276477 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:09.276439    4865 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 02:15:09.276669 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:09.276499    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2" podUID="746c1e17-96d9-40fd-8cce-8aebfbc17666" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 02:15:09.788691 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:09.788662    4865 scope.go:122] "RemoveContainer" containerID="92fdba7b77ad36ee25242602aeb3449a5865b028ea13e0c22909560961344eac"
Nov 04 02:15:09.789145 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:15:09.788939    4865 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld_odh-ai-gateway-infra(a76164fe-da41-44fa-b5c7-31cceee69d17)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld" podUID="a76164fe-da41-44fa-b5c7-31cceee69d17"
Nov 04 02:15:10.276265 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:10.276231    4865 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 02:15:10.276434 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:10.276311    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2" podUID="746c1e17-96d9-40fd-8cce-8aebfbc17666" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 02:15:11.276694 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:11.276654    4865 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 02:15:11.277054 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:11.276726    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2" podUID="746c1e17-96d9-40fd-8cce-8aebfbc17666" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 02:15:12.277098 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:12.277063    4865 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 02:15:12.277537 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:12.277152    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2" podUID="746c1e17-96d9-40fd-8cce-8aebfbc17666" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 02:15:13.276996 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:13.276966    4865 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 02:15:13.277175 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:13.277021    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2" podUID="746c1e17-96d9-40fd-8cce-8aebfbc17666" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 02:15:14.280831 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:14.280800    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:14.281198 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:14.281175    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:14.281963 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:14.281943    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:15:18.888910 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:18.888879    4865 scope.go:122] "RemoveContainer" containerID="92fdba7b77ad36ee25242602aeb3449a5865b028ea13e0c22909560961344eac"
Nov 04 02:15:19.853471 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:19.853440    4865 generic.go:356] "Generic (PLEG): container finished" podID="a76164fe-da41-44fa-b5c7-31cceee69d17" containerID="102ff0eb4f9dc30ed47114eca09b48bf40bd98fa19aa5a77e30e24b239fe052d" exitCode=60
Nov 04 02:15:19.853704 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:19.853518    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld" event={"ID":"a76164fe-da41-44fa-b5c7-31cceee69d17","Type":"ContainerDied","Data":"102ff0eb4f9dc30ed47114eca09b48bf40bd98fa19aa5a77e30e24b239fe052d"}
Nov 04 02:15:19.853704 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:19.853586    4865 scope.go:122] "RemoveContainer" containerID="92fdba7b77ad36ee25242602aeb3449a5865b028ea13e0c22909560961344eac"
Nov 04 02:15:19.854018 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:19.853998    4865 scope.go:122] "RemoveContainer" containerID="102ff0eb4f9dc30ed47114eca09b48bf40bd98fa19aa5a77e30e24b239fe052d"
Nov 04 02:15:19.854323 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:15:19.854296    4865 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld_odh-ai-gateway-infra(a76164fe-da41-44fa-b5c7-31cceee69d17)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld" podUID="a76164fe-da41-44fa-b5c7-31cceee69d17"
Nov 04 02:15:20.982444 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:20.982407    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld"]
Nov 04 02:15:21.127237 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:21.127213    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld"
Nov 04 02:15:21.261618 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:21.261542    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a76164fe-da41-44fa-b5c7-31cceee69d17-kube-api-access-wwdzs\" (UniqueName: \"kubernetes.io/projected/a76164fe-da41-44fa-b5c7-31cceee69d17-kube-api-access-wwdzs\") pod \"a76164fe-da41-44fa-b5c7-31cceee69d17\" (UID: \"a76164fe-da41-44fa-b5c7-31cceee69d17\") "
Nov 04 02:15:21.261821 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:21.261692    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a76164fe-da41-44fa-b5c7-31cceee69d17-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a76164fe-da41-44fa-b5c7-31cceee69d17-maas-api-service-ca\") pod \"a76164fe-da41-44fa-b5c7-31cceee69d17\" (UID: \"a76164fe-da41-44fa-b5c7-31cceee69d17\") "
Nov 04 02:15:21.262152 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:21.262115    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a76164fe-da41-44fa-b5c7-31cceee69d17-maas-api-service-ca" pod "a76164fe-da41-44fa-b5c7-31cceee69d17" (UID: "a76164fe-da41-44fa-b5c7-31cceee69d17"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:15:21.263835 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:21.263804    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a76164fe-da41-44fa-b5c7-31cceee69d17-kube-api-access-wwdzs" pod "a76164fe-da41-44fa-b5c7-31cceee69d17" (UID: "a76164fe-da41-44fa-b5c7-31cceee69d17"). InnerVolumeSpecName "kube-api-access-wwdzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:15:21.363091 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:21.363066    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwdzs\" (UniqueName: \"kubernetes.io/projected/a76164fe-da41-44fa-b5c7-31cceee69d17-kube-api-access-wwdzs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:21.363091 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:21.363091    4865 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a76164fe-da41-44fa-b5c7-31cceee69d17-maas-api-service-ca\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:21.868420 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:21.868384    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld" event={"ID":"a76164fe-da41-44fa-b5c7-31cceee69d17","Type":"ContainerDied","Data":"7607952220c7746fb11b0855f83a0eaeb3e52c335d92dfca8999d40afbdd6556"}
Nov 04 02:15:21.868634 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:21.868450    4865 scope.go:122] "RemoveContainer" containerID="102ff0eb4f9dc30ed47114eca09b48bf40bd98fa19aa5a77e30e24b239fe052d"
Nov 04 02:15:21.868634 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:21.868396    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld"
Nov 04 02:15:22.036264 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:22.036231    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld"]
Nov 04 02:15:22.087174 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:22.087128    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29e7e8-29895975-vhdld"]
Nov 04 02:15:22.895475 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:22.895434    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a76164fe-da41-44fa-b5c7-31cceee69d17" path="/var/lib/kubelet/pods/a76164fe-da41-44fa-b5c7-31cceee69d17/volumes"
Nov 04 02:15:28.404775 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:15:28.404737    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:15:33.243236 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:33.243199    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:15:37.011177 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:37.011149    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/2.log"
Nov 04 02:15:37.016758 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:37.016731    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/2.log"
Nov 04 02:15:41.533585 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.533549    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"]
Nov 04 02:15:41.534053 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.534039    4865 cpu_manager.go:414] "containerMap: removing container" podUID="a76164fe-da41-44fa-b5c7-31cceee69d17" containerName="cleanup"
Nov 04 02:15:41.534108 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.534054    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a76164fe-da41-44fa-b5c7-31cceee69d17" containerName="cleanup"
Nov 04 02:15:41.534108 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.534082    4865 cpu_manager.go:414] "containerMap: removing container" podUID="a76164fe-da41-44fa-b5c7-31cceee69d17" containerName="cleanup"
Nov 04 02:15:41.534108 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.534087    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a76164fe-da41-44fa-b5c7-31cceee69d17" containerName="cleanup"
Nov 04 02:15:41.534204 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.534171    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="a76164fe-da41-44fa-b5c7-31cceee69d17" containerName="cleanup"
Nov 04 02:15:41.534204 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.534178    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="a76164fe-da41-44fa-b5c7-31cceee69d17" containerName="cleanup"
Nov 04 02:15:41.538608 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.538592    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.615323 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.615275    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-istio-envoy\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.615480 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.615425    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgj7x\" (UniqueName: \"kubernetes.io/projected/3266fdea-562e-4e27-8b95-c1157b563921-kube-api-access-rgj7x\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.615523 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.615480    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-istio-data\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.615558 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.615538    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3266fdea-562e-4e27-8b95-c1157b563921-istiod-ca-cert\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.615592 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.615564    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3266fdea-562e-4e27-8b95-c1157b563921-istio-token\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.615637 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.615603    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3266fdea-562e-4e27-8b95-c1157b563921-istio-podinfo\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.615731 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.615709    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-credential-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.615776 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.615751    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-workload-certs\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.615824 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.615780    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-workload-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.640868 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.640832    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"]
Nov 04 02:15:41.717017 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.716984    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-workload-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.717193 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.717042    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-istio-envoy\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.717193 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.717157    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgj7x\" (UniqueName: \"kubernetes.io/projected/3266fdea-562e-4e27-8b95-c1157b563921-kube-api-access-rgj7x\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.717344 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.717211    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-istio-data\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.717344 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.717235    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3266fdea-562e-4e27-8b95-c1157b563921-istiod-ca-cert\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.717344 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.717264    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3266fdea-562e-4e27-8b95-c1157b563921-istio-token\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.717344 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.717316    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3266fdea-562e-4e27-8b95-c1157b563921-istio-podinfo\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.717555 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.717387    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-credential-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.717555 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.717416    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-workload-certs\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.717555 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.717456    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-workload-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.717953 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.717928    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-credential-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.718026 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.717956    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-workload-certs\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.718149 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.718121    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3266fdea-562e-4e27-8b95-c1157b563921-istiod-ca-cert\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.718323 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.718198    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-istio-data\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.719684 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.719666    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-istio-envoy\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.720304 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.720270    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3266fdea-562e-4e27-8b95-c1157b563921-istio-podinfo\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.779231 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.779200    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3266fdea-562e-4e27-8b95-c1157b563921-istio-token\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.779453 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.779434    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgj7x\" (UniqueName: \"kubernetes.io/projected/3266fdea-562e-4e27-8b95-c1157b563921-kube-api-access-rgj7x\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:41.852416 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:41.852348    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:42.280180 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:42.280153    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"]
Nov 04 02:15:42.280903 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:15:42.280878    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3266fdea_562e_4e27_8b95_c1157b563921.slice/crio-0b310f3621a5b04e7f2745f37985537b0bcc8d5d053574cb065ff27db7ed2dba WatchSource:0}: Error finding container 0b310f3621a5b04e7f2745f37985537b0bcc8d5d053574cb065ff27db7ed2dba: Status 404 returned error can't find the container with id 0b310f3621a5b04e7f2745f37985537b0bcc8d5d053574cb065ff27db7ed2dba
Nov 04 02:15:42.283107 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:42.283072    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:15:42.283193 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:42.283139    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:15:42.283193 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:42.283168    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:15:43.019738 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.019700    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh" event={"ID":"3266fdea-562e-4e27-8b95-c1157b563921","Type":"ContainerStarted","Data":"a76eea58e2114802ff0e889b1465b5e2c68ad143eb4089836c3ed97389937a4b"}
Nov 04 02:15:43.020176 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.019754    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh" event={"ID":"3266fdea-562e-4e27-8b95-c1157b563921","Type":"ContainerStarted","Data":"0b310f3621a5b04e7f2745f37985537b0bcc8d5d053574cb065ff27db7ed2dba"}
Nov 04 02:15:43.255620 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.255554    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh" podStartSLOduration=2.2555318189999998 podStartE2EDuration="2.255531819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:43.195031884 +0000 UTC m=+1206.421818552" watchObservedRunningTime="2026-11-04 02:15:43.255531819 +0000 UTC m=+1206.482318488"
Nov 04 02:15:43.578350 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.578313    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"]
Nov 04 02:15:43.578861 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.578846    4865 cpu_manager.go:414] "containerMap: removing container" podUID="a76164fe-da41-44fa-b5c7-31cceee69d17" containerName="cleanup"
Nov 04 02:15:43.578903 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.578863    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a76164fe-da41-44fa-b5c7-31cceee69d17" containerName="cleanup"
Nov 04 02:15:43.578964 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.578955    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="a76164fe-da41-44fa-b5c7-31cceee69d17" containerName="cleanup"
Nov 04 02:15:43.581247 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.581229    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:15:43.650453 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.650421    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-055a14-serving-cert\""
Nov 04 02:15:43.663009 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.662860    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"]
Nov 04 02:15:43.765031 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.764997    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlcbt\" (UniqueName: \"kubernetes.io/projected/cc3036dd-9c7a-4407-a710-696c360d2e43-kube-api-access-rlcbt\") pod \"maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn\" (UID: \"cc3036dd-9c7a-4407-a710-696c360d2e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:15:43.765211 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.765059    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc3036dd-9c7a-4407-a710-696c360d2e43-maas-api-tls\") pod \"maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn\" (UID: \"cc3036dd-9c7a-4407-a710-696c360d2e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:15:43.765252 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.765212    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc3036dd-9c7a-4407-a710-696c360d2e43-metrics-certs\") pod \"maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn\" (UID: \"cc3036dd-9c7a-4407-a710-696c360d2e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:15:43.765379 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.765360    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc3036dd-9c7a-4407-a710-696c360d2e43-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn\" (UID: \"cc3036dd-9c7a-4407-a710-696c360d2e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:15:43.877349 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.874018    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc3036dd-9c7a-4407-a710-696c360d2e43-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn\" (UID: \"cc3036dd-9c7a-4407-a710-696c360d2e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:15:43.877349 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.874122    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlcbt\" (UniqueName: \"kubernetes.io/projected/cc3036dd-9c7a-4407-a710-696c360d2e43-kube-api-access-rlcbt\") pod \"maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn\" (UID: \"cc3036dd-9c7a-4407-a710-696c360d2e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:15:43.877349 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.874186    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc3036dd-9c7a-4407-a710-696c360d2e43-maas-api-tls\") pod \"maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn\" (UID: \"cc3036dd-9c7a-4407-a710-696c360d2e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:15:43.877349 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.874240    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc3036dd-9c7a-4407-a710-696c360d2e43-metrics-certs\") pod \"maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn\" (UID: \"cc3036dd-9c7a-4407-a710-696c360d2e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:15:43.878327 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.878250    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:43.878580 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.878555    4865 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 04 02:15:43.878731 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.878705    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh" podUID="3266fdea-562e-4e27-8b95-c1157b563921" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 04 02:15:43.878859 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.878765    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc3036dd-9c7a-4407-a710-696c360d2e43-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn\" (UID: \"cc3036dd-9c7a-4407-a710-696c360d2e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:15:43.878965 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:15:43.878869    4865 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-serving-cert: secret "maas-api-e2e-extmod-b-055a14-serving-cert" not found
Nov 04 02:15:43.879097 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:15:43.879084    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc3036dd-9c7a-4407-a710-696c360d2e43-maas-api-tls podName:cc3036dd-9c7a-4407-a710-696c360d2e43 nodeName:}" failed. No retries permitted until 2026-11-04 02:15:44.379063356 +0000 UTC m=+1207.605850015 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cc3036dd-9c7a-4407-a710-696c360d2e43-maas-api-tls") pod "maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn" (UID: "cc3036dd-9c7a-4407-a710-696c360d2e43") : secret "maas-api-e2e-extmod-b-055a14-serving-cert" not found
Nov 04 02:15:43.888981 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.888927    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc3036dd-9c7a-4407-a710-696c360d2e43-metrics-certs\") pod \"maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn\" (UID: \"cc3036dd-9c7a-4407-a710-696c360d2e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:15:43.957963 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:43.957924    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlcbt\" (UniqueName: \"kubernetes.io/projected/cc3036dd-9c7a-4407-a710-696c360d2e43-kube-api-access-rlcbt\") pod \"maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn\" (UID: \"cc3036dd-9c7a-4407-a710-696c360d2e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:15:44.381590 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:44.381499    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc3036dd-9c7a-4407-a710-696c360d2e43-maas-api-tls\") pod \"maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn\" (UID: \"cc3036dd-9c7a-4407-a710-696c360d2e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:15:44.384139 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:44.384110    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc3036dd-9c7a-4407-a710-696c360d2e43-maas-api-tls\") pod \"maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn\" (UID: \"cc3036dd-9c7a-4407-a710-696c360d2e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:15:44.493339 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:44.493310    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:15:44.856706 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:44.856654    4865 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 04 02:15:44.856910 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:44.856714    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh" podUID="3266fdea-562e-4e27-8b95-c1157b563921" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 04 02:15:44.901035 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:44.901008    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"]
Nov 04 02:15:44.901801 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:15:44.901764    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc3036dd_9c7a_4407_a710_696c360d2e43.slice/crio-ed83d6694cfa4f4d2c5c9b3a349cddb621096f4717769bd3ccedbe7a4f482360 WatchSource:0}: Error finding container ed83d6694cfa4f4d2c5c9b3a349cddb621096f4717769bd3ccedbe7a4f482360: Status 404 returned error can't find the container with id ed83d6694cfa4f4d2c5c9b3a349cddb621096f4717769bd3ccedbe7a4f482360
Nov 04 02:15:45.055307 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:45.054605    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn" event={"ID":"cc3036dd-9c7a-4407-a710-696c360d2e43","Type":"ContainerStarted","Data":"ed83d6694cfa4f4d2c5c9b3a349cddb621096f4717769bd3ccedbe7a4f482360"}
Nov 04 02:15:45.857703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:45.857655    4865 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 04 02:15:45.858122 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:45.857738    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh" podUID="3266fdea-562e-4e27-8b95-c1157b563921" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 04 02:15:46.064548 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:46.064457    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn" event={"ID":"cc3036dd-9c7a-4407-a710-696c360d2e43","Type":"ContainerStarted","Data":"a9d38fae3738337f1d526c4dc718d77cdc1d00c4be0d09ebe9409a868093d31a"}
Nov 04 02:15:46.064548 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:46.064531    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:15:46.220306 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:46.220229    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn" podStartSLOduration=2.314411174 podStartE2EDuration="3.220214077s" totalImagesPullingTime="905.802903ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:46.218489535 +0000 UTC m=+1209.445276204" watchObservedRunningTime="2026-11-04 02:15:46.220214077 +0000 UTC m=+1209.447000746"
Nov 04 02:15:46.853414 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:46.853376    4865 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 04 02:15:46.853704 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:46.853675    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh" podUID="3266fdea-562e-4e27-8b95-c1157b563921" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 04 02:15:47.854629 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:47.854593    4865 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 04 02:15:47.855510 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:47.855467    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh" podUID="3266fdea-562e-4e27-8b95-c1157b563921" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 04 02:15:48.852654 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:48.852620    4865 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 04 02:15:48.852861 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:48.852683    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh" podUID="3266fdea-562e-4e27-8b95-c1157b563921" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 04 02:15:50.264364 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:50.262639    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:50.272401 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:50.272362    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:50.276647 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:50.276619    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:15:52.075799 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:52.075770    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:15:58.403589 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:15:58.403552    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:15:59.809945 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:15:59.809912    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:16:09.245885 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:09.245852    4865 generic.go:356] "Generic (PLEG): container finished" podID="e211ff43-fa6b-4912-9c5b-bc5cec322b03" containerID="658d4407678b49127be650b8863da502047b56d33b7729eb7df4914af010ddfe" exitCode=0
Nov 04 02:16:09.246349 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:09.245921    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l" event={"ID":"e211ff43-fa6b-4912-9c5b-bc5cec322b03","Type":"ContainerDied","Data":"658d4407678b49127be650b8863da502047b56d33b7729eb7df4914af010ddfe"}
Nov 04 02:16:09.246349 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:09.246300    4865 scope.go:122] "RemoveContainer" containerID="658d4407678b49127be650b8863da502047b56d33b7729eb7df4914af010ddfe"
Nov 04 02:16:09.247725 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:09.247695    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:16:09.247836 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:09.247778    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:16:09.247836 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:09.247829    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:16:10.265168 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:10.265134    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l" event={"ID":"e211ff43-fa6b-4912-9c5b-bc5cec322b03","Type":"ContainerStarted","Data":"fefcf305f9fc9103f75e5c09f87e53e8cb7ab398ac866b53e550d574755101a5"}
Nov 04 02:16:10.984703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:10.984670    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:16:10.984893 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:10.984836    4865 patch_prober.go:29] interesting pod/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 04 02:16:10.984941 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:10.984890    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l" podUID="e211ff43-fa6b-4912-9c5b-bc5cec322b03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 04 02:16:11.291975 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.291893    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"]
Nov 04 02:16:11.294730 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.294710    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.355304 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.350609    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-dab533-openshift-default-dockercfg-tnhqz\""
Nov 04 02:16:11.404227 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.404182    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87f5ff3d-b612-4b82-a700-221881b22907-istio-podinfo\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.404409 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.404258    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87f5ff3d-b612-4b82-a700-221881b22907-istiod-ca-cert\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.404463 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.404417    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-workload-socket\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.404500 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.404459    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-istio-envoy\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.404500 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.404489    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-istio-data\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.404576 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.404514    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhdqf\" (UniqueName: \"kubernetes.io/projected/87f5ff3d-b612-4b82-a700-221881b22907-kube-api-access-jhdqf\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.404576 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.404561    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-workload-certs\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.404658 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.404615    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-credential-socket\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.404695 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.404659    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"]
Nov 04 02:16:11.405046 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.405019    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87f5ff3d-b612-4b82-a700-221881b22907-istio-token\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.512313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.511988    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-istio-data\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.512313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.512047    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhdqf\" (UniqueName: \"kubernetes.io/projected/87f5ff3d-b612-4b82-a700-221881b22907-kube-api-access-jhdqf\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.512313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.512084    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-workload-certs\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.512313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.512137    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-credential-socket\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.512313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.512168    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87f5ff3d-b612-4b82-a700-221881b22907-istio-token\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.512313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.512303    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87f5ff3d-b612-4b82-a700-221881b22907-istio-podinfo\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.512779 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.512347    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87f5ff3d-b612-4b82-a700-221881b22907-istiod-ca-cert\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.512779 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.512452    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-workload-socket\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.512779 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.512494    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-istio-envoy\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.532369 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.532335    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-credential-socket\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.542434 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.533137    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-istio-data\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.553160 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.541670    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-workload-certs\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.553677 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.542304    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87f5ff3d-b612-4b82-a700-221881b22907-istiod-ca-cert\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.553832 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.543035    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-workload-socket\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.571304 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.558808    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-istio-envoy\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.572051 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.572020    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87f5ff3d-b612-4b82-a700-221881b22907-istio-podinfo\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.601927 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.601893    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhdqf\" (UniqueName: \"kubernetes.io/projected/87f5ff3d-b612-4b82-a700-221881b22907-kube-api-access-jhdqf\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.604463 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.604434    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87f5ff3d-b612-4b82-a700-221881b22907-istio-token\") pod \"e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") " pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.607733 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.607708    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:11.984703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.984618    4865 patch_prober.go:29] interesting pod/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 04 02:16:11.984703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:11.984678    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l" podUID="e211ff43-fa6b-4912-9c5b-bc5cec322b03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 04 02:16:12.062433 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:12.062405    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"]
Nov 04 02:16:12.063380 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:16:12.063356    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87f5ff3d_b612_4b82_a700_221881b22907.slice/crio-d636b67c4f377bea341cf1d268743ff77b8cb0ac86da674461f42e3f18ef9a78 WatchSource:0}: Error finding container d636b67c4f377bea341cf1d268743ff77b8cb0ac86da674461f42e3f18ef9a78: Status 404 returned error can't find the container with id d636b67c4f377bea341cf1d268743ff77b8cb0ac86da674461f42e3f18ef9a78
Nov 04 02:16:12.066231 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:12.066199    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:16:12.066326 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:12.066266    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:16:12.066377 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:12.066344    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:16:12.277678 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:12.277591    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26" event={"ID":"87f5ff3d-b612-4b82-a700-221881b22907","Type":"ContainerStarted","Data":"8d1b814eae74f3709e5a75fd6b63af31e1b508ead98fcb8f78fb1289afa18c41"}
Nov 04 02:16:12.277678 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:12.277655    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26" event={"ID":"87f5ff3d-b612-4b82-a700-221881b22907","Type":"ContainerStarted","Data":"d636b67c4f377bea341cf1d268743ff77b8cb0ac86da674461f42e3f18ef9a78"}
Nov 04 02:16:12.449307 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:12.446258    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26" podStartSLOduration=1.4462368030000001 podStartE2EDuration="1.446236803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:12.435851949 +0000 UTC m=+1235.662638634" watchObservedRunningTime="2026-11-04 02:16:12.446236803 +0000 UTC m=+1235.673023472"
Nov 04 02:16:12.616033 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:12.612614    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:12.984453 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:12.984420    4865 patch_prober.go:29] interesting pod/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 04 02:16:12.984628 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:12.984476    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l" podUID="e211ff43-fa6b-4912-9c5b-bc5cec322b03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 04 02:16:13.614509 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:13.614474    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:13.984268 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:13.984236    4865 patch_prober.go:29] interesting pod/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 04 02:16:13.984432 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:13.984307    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l" podUID="e211ff43-fa6b-4912-9c5b-bc5cec322b03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 04 02:16:14.289561 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:14.289466    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:14.290487 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:14.290466    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:16:14.985113 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:14.985079    4865 patch_prober.go:29] interesting pod/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 04 02:16:14.985545 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:14.985133    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l" podUID="e211ff43-fa6b-4912-9c5b-bc5cec322b03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 04 02:16:15.988632 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:15.988601    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:16:15.989068 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:15.988903    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:16:15.989604 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:15.989584    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:16:28.406639 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:16:28.406595    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:16:36.423099 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:36.423008    4865 generic.go:356] "Generic (PLEG): container finished" podID="91d9868b-f91c-424b-8671-f2973afdfbfa" containerID="938e6fe08c4824de9f1c46784d8e7b6509d0d3e2f3052fdb91bc1f8f601c09c3" exitCode=0
Nov 04 02:16:36.423099 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:36.423081    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql" event={"ID":"91d9868b-f91c-424b-8671-f2973afdfbfa","Type":"ContainerDied","Data":"938e6fe08c4824de9f1c46784d8e7b6509d0d3e2f3052fdb91bc1f8f601c09c3"}
Nov 04 02:16:36.423557 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:36.423488    4865 scope.go:122] "RemoveContainer" containerID="938e6fe08c4824de9f1c46784d8e7b6509d0d3e2f3052fdb91bc1f8f601c09c3"
Nov 04 02:16:36.424824 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:36.424796    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:16:36.424876 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:36.424866    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:16:36.424915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:36.424897    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:16:36.758401 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:36.758325    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:16:37.430566 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:37.430531    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql" event={"ID":"91d9868b-f91c-424b-8671-f2973afdfbfa","Type":"ContainerStarted","Data":"2c0a1f07f8442adb67dbf1ba8915f64876e27b2d2ede981f13e401b2f255fc9c"}
Nov 04 02:16:38.273027 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.272988    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"]
Nov 04 02:16:38.277168 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.277145    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.323799 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.323763    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"]
Nov 04 02:16:38.327531 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.327510    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.330169 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.330150    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:16:38.349305 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.331382    4865 patch_prober.go:29] interesting pod/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 02:16:38.354318 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.354268    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql" podUID="91d9868b-f91c-424b-8671-f2973afdfbfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 02:16:38.354571 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.353494    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wgllv\""
Nov 04 02:16:38.372878 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.372855    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffac90da-23d5-4cf1-9dcf-f543bc11736b-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.372985 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.372905    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffac90da-23d5-4cf1-9dcf-f543bc11736b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.373059 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.373031    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15993036-6c89-4844-b870-ccb12cfc744a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.373113 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.373080    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffac90da-23d5-4cf1-9dcf-f543bc11736b-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.373189 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.373166    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffac90da-23d5-4cf1-9dcf-f543bc11736b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.373336 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.373220    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15993036-6c89-4844-b870-ccb12cfc744a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.373421 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.373347    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvcpj\" (UniqueName: \"kubernetes.io/projected/15993036-6c89-4844-b870-ccb12cfc744a-kube-api-access-zvcpj\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.374393 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.374372    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2lbm\" (UniqueName: \"kubernetes.io/projected/ffac90da-23d5-4cf1-9dcf-f543bc11736b-kube-api-access-p2lbm\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.374500 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.374444    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffac90da-23d5-4cf1-9dcf-f543bc11736b-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.374612 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.374590    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15993036-6c89-4844-b870-ccb12cfc744a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.374673 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.374620    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15993036-6c89-4844-b870-ccb12cfc744a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.374673 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.374644    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15993036-6c89-4844-b870-ccb12cfc744a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.374673 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.374669    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffac90da-23d5-4cf1-9dcf-f543bc11736b-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.374810 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.374686    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15993036-6c89-4844-b870-ccb12cfc744a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.374810 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.374716    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15993036-6c89-4844-b870-ccb12cfc744a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.374810 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.374768    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffac90da-23d5-4cf1-9dcf-f543bc11736b-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.374810 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.374783    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffac90da-23d5-4cf1-9dcf-f543bc11736b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.374810 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.374799    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15993036-6c89-4844-b870-ccb12cfc744a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.378803 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.378772    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"]
Nov 04 02:16:38.389887 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.389858    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"]
Nov 04 02:16:38.476098 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476067    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffac90da-23d5-4cf1-9dcf-f543bc11736b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.476636 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476119    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15993036-6c89-4844-b870-ccb12cfc744a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.476636 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476151    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffac90da-23d5-4cf1-9dcf-f543bc11736b-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.476636 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476180    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffac90da-23d5-4cf1-9dcf-f543bc11736b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.476636 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476210    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15993036-6c89-4844-b870-ccb12cfc744a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.476636 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476243    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvcpj\" (UniqueName: \"kubernetes.io/projected/15993036-6c89-4844-b870-ccb12cfc744a-kube-api-access-zvcpj\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.476636 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476322    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2lbm\" (UniqueName: \"kubernetes.io/projected/ffac90da-23d5-4cf1-9dcf-f543bc11736b-kube-api-access-p2lbm\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.476636 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476398    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffac90da-23d5-4cf1-9dcf-f543bc11736b-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.476636 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476538    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15993036-6c89-4844-b870-ccb12cfc744a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.476636 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476566    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15993036-6c89-4844-b870-ccb12cfc744a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.476636 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476603    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15993036-6c89-4844-b870-ccb12cfc744a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.476636 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476617    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffac90da-23d5-4cf1-9dcf-f543bc11736b-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.476636 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476644    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffac90da-23d5-4cf1-9dcf-f543bc11736b-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.477313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476689    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15993036-6c89-4844-b870-ccb12cfc744a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.477313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476727    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15993036-6c89-4844-b870-ccb12cfc744a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.477313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476789    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffac90da-23d5-4cf1-9dcf-f543bc11736b-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.477313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476813    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffac90da-23d5-4cf1-9dcf-f543bc11736b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.477313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476838    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15993036-6c89-4844-b870-ccb12cfc744a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.477313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476862    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffac90da-23d5-4cf1-9dcf-f543bc11736b-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.477313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476882    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffac90da-23d5-4cf1-9dcf-f543bc11736b-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.477313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.476982    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffac90da-23d5-4cf1-9dcf-f543bc11736b-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.477313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.477100    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffac90da-23d5-4cf1-9dcf-f543bc11736b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.477313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.477141    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffac90da-23d5-4cf1-9dcf-f543bc11736b-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.477313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.477223    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15993036-6c89-4844-b870-ccb12cfc744a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.477313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.477311    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15993036-6c89-4844-b870-ccb12cfc744a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.477949 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.477558    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15993036-6c89-4844-b870-ccb12cfc744a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.477949 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.477621    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15993036-6c89-4844-b870-ccb12cfc744a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.477949 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.477741    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15993036-6c89-4844-b870-ccb12cfc744a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.478895 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.478870    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15993036-6c89-4844-b870-ccb12cfc744a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.479381 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.479364    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffac90da-23d5-4cf1-9dcf-f543bc11736b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.479465 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.479443    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffac90da-23d5-4cf1-9dcf-f543bc11736b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.479645 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.479629    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15993036-6c89-4844-b870-ccb12cfc744a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.558454 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.558368    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2lbm\" (UniqueName: \"kubernetes.io/projected/ffac90da-23d5-4cf1-9dcf-f543bc11736b-kube-api-access-p2lbm\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.558704 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.558676    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15993036-6c89-4844-b870-ccb12cfc744a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.559026 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.559006    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvcpj\" (UniqueName: \"kubernetes.io/projected/15993036-6c89-4844-b870-ccb12cfc744a-kube-api-access-zvcpj\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw\" (UID: \"15993036-6c89-4844-b870-ccb12cfc744a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:38.559090 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.559015    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffac90da-23d5-4cf1-9dcf-f543bc11736b-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd\" (UID: \"ffac90da-23d5-4cf1-9dcf-f543bc11736b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.591177 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.591151    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:38.662835 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:38.662523    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:39.057137 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.057109    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"]
Nov 04 02:16:39.065392 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:16:39.065362    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffac90da_23d5_4cf1_9dcf_f543bc11736b.slice/crio-2cdcfe08d88b541d6daa4431f2b8eea9be12fd66f96e8cb53eca909d23df86dd WatchSource:0}: Error finding container 2cdcfe08d88b541d6daa4431f2b8eea9be12fd66f96e8cb53eca909d23df86dd: Status 404 returned error can't find the container with id 2cdcfe08d88b541d6daa4431f2b8eea9be12fd66f96e8cb53eca909d23df86dd
Nov 04 02:16:39.068000 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.067970    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:16:39.068077 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.068033    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:16:39.068077 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.068062    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:16:39.096977 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.096950    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"]
Nov 04 02:16:39.100494 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:16:39.100469    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15993036_6c89_4844_b870_ccb12cfc744a.slice/crio-ac6d073876b77c203d73dc62d190ec4c479d6890e3a2c4a905b93fdd7abe070e WatchSource:0}: Error finding container ac6d073876b77c203d73dc62d190ec4c479d6890e3a2c4a905b93fdd7abe070e: Status 404 returned error can't find the container with id ac6d073876b77c203d73dc62d190ec4c479d6890e3a2c4a905b93fdd7abe070e
Nov 04 02:16:39.102868 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.102835    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:16:39.102926 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.102916    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:16:39.102976 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.102960    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:16:39.330594 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.330516    4865 patch_prober.go:29] interesting pod/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 02:16:39.330594 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.330572    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql" podUID="91d9868b-f91c-424b-8671-f2973afdfbfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 02:16:39.473537 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.473499    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd" event={"ID":"ffac90da-23d5-4cf1-9dcf-f543bc11736b","Type":"ContainerStarted","Data":"62bd5e6c99e62a4a9e01c59c38e39b1c8686a3cb2af2c1fd6df452972ae15139"}
Nov 04 02:16:39.473688 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.473562    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd" event={"ID":"ffac90da-23d5-4cf1-9dcf-f543bc11736b","Type":"ContainerStarted","Data":"2cdcfe08d88b541d6daa4431f2b8eea9be12fd66f96e8cb53eca909d23df86dd"}
Nov 04 02:16:39.475409 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.475379    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw" event={"ID":"15993036-6c89-4844-b870-ccb12cfc744a","Type":"ContainerStarted","Data":"f81420f5559e69a23212ea3c7fca4eb8e66f9247c4550ab553bf4470dfff4014"}
Nov 04 02:16:39.475500 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.475444    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw" event={"ID":"15993036-6c89-4844-b870-ccb12cfc744a","Type":"ContainerStarted","Data":"ac6d073876b77c203d73dc62d190ec4c479d6890e3a2c4a905b93fdd7abe070e"}
Nov 04 02:16:39.592495 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.592000    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:39.602531 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.602468    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd" podStartSLOduration=1.602451023 podStartE2EDuration="1.602451023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:39.599581822 +0000 UTC m=+1262.826368491" watchObservedRunningTime="2026-11-04 02:16:39.602451023 +0000 UTC m=+1262.829237693"
Nov 04 02:16:39.664401 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.664365    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:39.725264 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:39.725204    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw" podStartSLOduration=1.725189648 podStartE2EDuration="1.725189648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:39.722926828 +0000 UTC m=+1262.949713489" watchObservedRunningTime="2026-11-04 02:16:39.725189648 +0000 UTC m=+1262.951976316"
Nov 04 02:16:40.349306 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:40.347251    4865 patch_prober.go:29] interesting pod/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 02:16:40.349306 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:40.347333    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql" podUID="91d9868b-f91c-424b-8671-f2973afdfbfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 02:16:40.609368 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:40.608493    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 02:16:40.609368 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:40.608567    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd" podUID="ffac90da-23d5-4cf1-9dcf-f543bc11736b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 02:16:40.666307 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:40.666206    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 02:16:40.666511 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:40.666320    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw" podUID="15993036-6c89-4844-b870-ccb12cfc744a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 02:16:41.331031 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:41.330974    4865 patch_prober.go:29] interesting pod/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 02:16:41.331239 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:41.331054    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql" podUID="91d9868b-f91c-424b-8671-f2973afdfbfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 02:16:41.592370 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:41.591644    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 02:16:41.592370 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:41.591720    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd" podUID="ffac90da-23d5-4cf1-9dcf-f543bc11736b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 02:16:41.664307 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:41.663629    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 02:16:41.664307 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:41.663696    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw" podUID="15993036-6c89-4844-b870-ccb12cfc744a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 02:16:42.341309 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:42.331166    4865 patch_prober.go:29] interesting pod/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 02:16:42.341309 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:42.331225    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql" podUID="91d9868b-f91c-424b-8671-f2973afdfbfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 02:16:42.592914 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:42.592806    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 02:16:42.593210 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:42.593160    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd" podUID="ffac90da-23d5-4cf1-9dcf-f543bc11736b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 02:16:42.663074 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:42.663000    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 02:16:42.663469 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:42.663389    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw" podUID="15993036-6c89-4844-b870-ccb12cfc744a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 02:16:43.356306 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:43.355906    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:16:43.360304 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:43.357322    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:16:43.380341 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:43.366487    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:16:43.591653 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:43.591621    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 02:16:43.591919 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:43.591890    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd" podUID="ffac90da-23d5-4cf1-9dcf-f543bc11736b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 02:16:43.669716 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:43.669639    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 02:16:43.669967 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:43.669939    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw" podUID="15993036-6c89-4844-b870-ccb12cfc744a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 02:16:44.591815 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:44.591776    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 02:16:44.592328 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:44.591837    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd" podUID="ffac90da-23d5-4cf1-9dcf-f543bc11736b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 02:16:44.670335 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:44.663387    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 02:16:44.670335 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:44.663453    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw" podUID="15993036-6c89-4844-b870-ccb12cfc744a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 02:16:45.591738 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:45.591696    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 02:16:45.591938 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:45.591759    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd" podUID="ffac90da-23d5-4cf1-9dcf-f543bc11736b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 02:16:45.663518 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:45.663486    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 02:16:45.663701 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:45.663546    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw" podUID="15993036-6c89-4844-b870-ccb12cfc744a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 02:16:46.591772 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:46.591731    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 02:16:46.591951 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:46.591808    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd" podUID="ffac90da-23d5-4cf1-9dcf-f543bc11736b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 02:16:46.663501 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:46.663464    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 02:16:46.663673 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:46.663523    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw" podUID="15993036-6c89-4844-b870-ccb12cfc744a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 02:16:48.472244 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:48.472210    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:48.473079 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:48.473055    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:48.474458 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:48.474435    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd"
Nov 04 02:16:48.607203 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:48.607167    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:48.607634 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:48.607607    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:48.608322 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:48.608276    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw"
Nov 04 02:16:53.279389 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:53.279358    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:16:58.403085 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:16:58.403050    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:16:59.980624 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:59.980588    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"]
Nov 04 02:16:59.984887 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:16:59.984867    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:17:00.020139 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:00.020099    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf428\" (UniqueName: \"kubernetes.io/projected/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-kube-api-access-mf428\") pod \"maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv\" (UID: \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:17:00.020345 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:00.020168    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-metrics-certs\") pod \"maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv\" (UID: \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:17:00.020345 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:00.020315    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-maas-api-tls\") pod \"maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv\" (UID: \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:17:00.020436 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:00.020376    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv\" (UID: \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:17:00.093064 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:00.093030    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-dab533-serving-cert\""
Nov 04 02:17:00.095777 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:00.095750    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"]
Nov 04 02:17:00.121299 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:00.121174    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf428\" (UniqueName: \"kubernetes.io/projected/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-kube-api-access-mf428\") pod \"maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv\" (UID: \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:17:00.121474 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:00.121385    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-metrics-certs\") pod \"maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv\" (UID: \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:17:00.121557 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:00.121488    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-maas-api-tls\") pod \"maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv\" (UID: \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:17:00.121622 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:00.121554    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv\" (UID: \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:17:00.121622 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:17:00.121577    4865 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-serving-cert: secret "maas-api-e2e-extmod-a-dab533-serving-cert" not found
Nov 04 02:17:00.121726 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:17:00.121661    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-maas-api-tls podName:134d6dc7-0882-4fe3-af67-d7ac7dc05e82 nodeName:}" failed. No retries permitted until 2026-11-04 02:17:00.621638978 +0000 UTC m=+1283.848425625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-maas-api-tls") pod "maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv" (UID: "134d6dc7-0882-4fe3-af67-d7ac7dc05e82") : secret "maas-api-e2e-extmod-a-dab533-serving-cert" not found
Nov 04 02:17:00.122468 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:00.122442    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv\" (UID: \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:17:00.124086 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:00.124059    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-metrics-certs\") pod \"maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv\" (UID: \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:17:00.183725 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:00.183692    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf428\" (UniqueName: \"kubernetes.io/projected/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-kube-api-access-mf428\") pod \"maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv\" (UID: \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:17:00.630872 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:00.630842    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-maas-api-tls\") pod \"maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv\" (UID: \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:17:00.633240 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:00.633214    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-maas-api-tls\") pod \"maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv\" (UID: \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:17:00.899956 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:00.899861    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:17:01.278993 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:01.278962    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"]
Nov 04 02:17:01.280250 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:17:01.280218    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod134d6dc7_0882_4fe3_af67_d7ac7dc05e82.slice/crio-2046103f945b2d0fc5c8a518874e534576b50b0e0fa64dc2a17815b7f7d5ff6f WatchSource:0}: Error finding container 2046103f945b2d0fc5c8a518874e534576b50b0e0fa64dc2a17815b7f7d5ff6f: Status 404 returned error can't find the container with id 2046103f945b2d0fc5c8a518874e534576b50b0e0fa64dc2a17815b7f7d5ff6f
Nov 04 02:17:01.707589 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:01.707555    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv" event={"ID":"134d6dc7-0882-4fe3-af67-d7ac7dc05e82","Type":"ContainerStarted","Data":"2046103f945b2d0fc5c8a518874e534576b50b0e0fa64dc2a17815b7f7d5ff6f"}
Nov 04 02:17:02.722310 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:02.722248    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv" event={"ID":"134d6dc7-0882-4fe3-af67-d7ac7dc05e82","Type":"ContainerStarted","Data":"eb45a9da2357ff57b92a696a76c4652dc3da7e68243c31b4ba4d2b009f2c9272"}
Nov 04 02:17:02.722798 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:02.722505    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:17:02.844130 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:02.844066    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv" podStartSLOduration=2.925959774 podStartE2EDuration="3.844052336s" totalImagesPullingTime="918.092562ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:17:02.841891607 +0000 UTC m=+1286.068678276" watchObservedRunningTime="2026-11-04 02:17:02.844052336 +0000 UTC m=+1286.070839004"
Nov 04 02:17:08.734109 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:08.734075    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:17:27.515249 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:27.515211    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"]
Nov 04 02:17:27.565635 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:27.565604    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"]
Nov 04 02:17:27.568295 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:27.568243    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"]
Nov 04 02:17:28.404193 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:17:28.404154    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:17:42.536236 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.536189    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq" podUID="6f585314-0bb4-48b5-9b4d-80a4e244089c" containerName="main" containerID="cri-o://7833b88efd58ca5cdfeaf30bc1eaf430db6da8c107bd4a7165317f17e7a860f1" gracePeriod=45
Nov 04 02:17:42.586507 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.586465    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl" podUID="cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5" containerName="main" containerID="cri-o://e1c0aa15d604bcf00bb651c1db3cf63b705184ce6536367912981f0cc63ca7cc" gracePeriod=45
Nov 04 02:17:42.587580 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.587532    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v" podUID="4dfcdafc-3763-472d-b34f-1783a66e838c" containerName="main" containerID="cri-o://2cc93dbded9dbbd71ea85d8c1bb18370fddc73afad24b1c9c9085d2a9e3e86d4" gracePeriod=45
Nov 04 02:17:42.883173 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.883147    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:17:42.891417 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.891388    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:17:42.895380 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.895357    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:17:42.940856 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.940824    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-home\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-home\") pod \"6f585314-0bb4-48b5-9b4d-80a4e244089c\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") "
Nov 04 02:17:42.940856 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.940864    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-home\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-home\") pod \"4dfcdafc-3763-472d-b34f-1783a66e838c\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") "
Nov 04 02:17:42.941149 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.940893    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-tmp-dir\") pod \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") "
Nov 04 02:17:42.941149 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.940924    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6f585314-0bb4-48b5-9b4d-80a4e244089c-tls-certs\" (UniqueName: \"kubernetes.io/secret/6f585314-0bb4-48b5-9b4d-80a4e244089c-tls-certs\") pod \"6f585314-0bb4-48b5-9b4d-80a4e244089c\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") "
Nov 04 02:17:42.941149 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.940967    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-tmp-dir\") pod \"6f585314-0bb4-48b5-9b4d-80a4e244089c\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") "
Nov 04 02:17:42.941149 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941018    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-home\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-home\") pod \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") "
Nov 04 02:17:42.941149 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941045    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-dshm\") pod \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") "
Nov 04 02:17:42.941149 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941080    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-model-cache\") pod \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") "
Nov 04 02:17:42.941149 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941145    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4dfcdafc-3763-472d-b34f-1783a66e838c-tls-certs\" (UniqueName: \"kubernetes.io/secret/4dfcdafc-3763-472d-b34f-1783a66e838c-tls-certs\") pod \"4dfcdafc-3763-472d-b34f-1783a66e838c\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") "
Nov 04 02:17:42.941601 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941155    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-home" pod "4dfcdafc-3763-472d-b34f-1783a66e838c" (UID: "4dfcdafc-3763-472d-b34f-1783a66e838c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:17:42.941601 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941161    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-home" pod "6f585314-0bb4-48b5-9b4d-80a4e244089c" (UID: "6f585314-0bb4-48b5-9b4d-80a4e244089c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:17:42.941601 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941184    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-tmp-dir\") pod \"4dfcdafc-3763-472d-b34f-1783a66e838c\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") "
Nov 04 02:17:42.941601 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941306    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-tmp-dir" pod "6f585314-0bb4-48b5-9b4d-80a4e244089c" (UID: "6f585314-0bb4-48b5-9b4d-80a4e244089c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:17:42.941601 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941359    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-model-cache\") pod \"4dfcdafc-3763-472d-b34f-1783a66e838c\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") "
Nov 04 02:17:42.941601 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941360    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-home" pod "cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5" (UID: "cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:17:42.941601 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941401    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-kube-api-access-9bpcm\" (UniqueName: \"kubernetes.io/projected/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-kube-api-access-9bpcm\") pod \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") "
Nov 04 02:17:42.941601 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941410    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-model-cache" pod "cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5" (UID: "cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:17:42.941601 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941452    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-tls-certs\" (UniqueName: \"kubernetes.io/secret/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-tls-certs\") pod \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\" (UID: \"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5\") "
Nov 04 02:17:42.941601 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941517    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4dfcdafc-3763-472d-b34f-1783a66e838c-kube-api-access-lqfld\" (UniqueName: \"kubernetes.io/projected/4dfcdafc-3763-472d-b34f-1783a66e838c-kube-api-access-lqfld\") pod \"4dfcdafc-3763-472d-b34f-1783a66e838c\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") "
Nov 04 02:17:42.941601 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941546    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-dshm\") pod \"4dfcdafc-3763-472d-b34f-1783a66e838c\" (UID: \"4dfcdafc-3763-472d-b34f-1783a66e838c\") "
Nov 04 02:17:42.941601 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941556    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-tmp-dir" pod "4dfcdafc-3763-472d-b34f-1783a66e838c" (UID: "4dfcdafc-3763-472d-b34f-1783a66e838c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:17:42.941601 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941601    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-model-cache\") pod \"6f585314-0bb4-48b5-9b4d-80a4e244089c\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") "
Nov 04 02:17:42.942499 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941632    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-dshm\") pod \"6f585314-0bb4-48b5-9b4d-80a4e244089c\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") "
Nov 04 02:17:42.942499 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941683    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f585314-0bb4-48b5-9b4d-80a4e244089c-kube-api-access-jcp4n\" (UniqueName: \"kubernetes.io/projected/6f585314-0bb4-48b5-9b4d-80a4e244089c-kube-api-access-jcp4n\") pod \"6f585314-0bb4-48b5-9b4d-80a4e244089c\" (UID: \"6f585314-0bb4-48b5-9b4d-80a4e244089c\") "
Nov 04 02:17:42.942499 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.942276    4865 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-tmp-dir\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:42.942499 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.942328    4865 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-home\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:42.942499 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.942342    4865 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-home\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:42.942499 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.942355    4865 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-tmp-dir\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:42.942499 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.942367    4865 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-home\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:42.942499 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.942379    4865 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-model-cache\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:42.944651 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941632    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-model-cache" pod "4dfcdafc-3763-472d-b34f-1783a66e838c" (UID: "4dfcdafc-3763-472d-b34f-1783a66e838c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:17:42.944651 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941775    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-tmp-dir" pod "cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5" (UID: "cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:17:42.944807 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.941957    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-model-cache" pod "6f585314-0bb4-48b5-9b4d-80a4e244089c" (UID: "6f585314-0bb4-48b5-9b4d-80a4e244089c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:17:42.944807 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.943921    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f585314-0bb4-48b5-9b4d-80a4e244089c-tls-certs" pod "6f585314-0bb4-48b5-9b4d-80a4e244089c" (UID: "6f585314-0bb4-48b5-9b4d-80a4e244089c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:17:42.944807 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.944052    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-kube-api-access-9bpcm" pod "cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5" (UID: "cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5"). InnerVolumeSpecName "kube-api-access-9bpcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:17:42.944807 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.944614    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-tls-certs" pod "cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5" (UID: "cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:17:42.944807 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.944691    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f585314-0bb4-48b5-9b4d-80a4e244089c-kube-api-access-jcp4n" pod "6f585314-0bb4-48b5-9b4d-80a4e244089c" (UID: "6f585314-0bb4-48b5-9b4d-80a4e244089c"). InnerVolumeSpecName "kube-api-access-jcp4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:17:42.944807 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.944732    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dfcdafc-3763-472d-b34f-1783a66e838c-tls-certs" pod "4dfcdafc-3763-472d-b34f-1783a66e838c" (UID: "4dfcdafc-3763-472d-b34f-1783a66e838c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:17:42.945005 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.944810    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dfcdafc-3763-472d-b34f-1783a66e838c-kube-api-access-lqfld" pod "4dfcdafc-3763-472d-b34f-1783a66e838c" (UID: "4dfcdafc-3763-472d-b34f-1783a66e838c"). InnerVolumeSpecName "kube-api-access-lqfld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:17:42.945005 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.944883    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-dshm" pod "4dfcdafc-3763-472d-b34f-1783a66e838c" (UID: "4dfcdafc-3763-472d-b34f-1783a66e838c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:17:42.945085 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.945071    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-dshm" pod "cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5" (UID: "cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:17:42.945941 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:42.945922    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-dshm" pod "6f585314-0bb4-48b5-9b4d-80a4e244089c" (UID: "6f585314-0bb4-48b5-9b4d-80a4e244089c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:17:43.043650 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.043606    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bpcm\" (UniqueName: \"kubernetes.io/projected/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-kube-api-access-9bpcm\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:43.043650 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.043642    4865 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-tls-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:43.043650 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.043652    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqfld\" (UniqueName: \"kubernetes.io/projected/4dfcdafc-3763-472d-b34f-1783a66e838c-kube-api-access-lqfld\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:43.043650 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.043661    4865 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-dshm\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:43.044021 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.043670    4865 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-model-cache\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:43.044021 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.043680    4865 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f585314-0bb4-48b5-9b4d-80a4e244089c-dshm\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:43.044021 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.043689    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcp4n\" (UniqueName: \"kubernetes.io/projected/6f585314-0bb4-48b5-9b4d-80a4e244089c-kube-api-access-jcp4n\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:43.044021 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.043699    4865 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-tmp-dir\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:43.044021 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.043707    4865 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f585314-0bb4-48b5-9b4d-80a4e244089c-tls-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:43.044021 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.043714    4865 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5-dshm\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:43.044021 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.043723    4865 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dfcdafc-3763-472d-b34f-1783a66e838c-tls-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:43.044021 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.043731    4865 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dfcdafc-3763-472d-b34f-1783a66e838c-model-cache\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:43.098780 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.098676    4865 generic.go:356] "Generic (PLEG): container finished" podID="4dfcdafc-3763-472d-b34f-1783a66e838c" containerID="2cc93dbded9dbbd71ea85d8c1bb18370fddc73afad24b1c9c9085d2a9e3e86d4" exitCode=0
Nov 04 02:17:43.098780 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.098729    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"
Nov 04 02:17:43.098780 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.098759    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v" event={"ID":"4dfcdafc-3763-472d-b34f-1783a66e838c","Type":"ContainerDied","Data":"2cc93dbded9dbbd71ea85d8c1bb18370fddc73afad24b1c9c9085d2a9e3e86d4"}
Nov 04 02:17:43.099041 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.098824    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v" event={"ID":"4dfcdafc-3763-472d-b34f-1783a66e838c","Type":"ContainerDied","Data":"78916cc8350ce3a13e4395f7429439981a452fdb417ac961b2ad84afb11dd6ec"}
Nov 04 02:17:43.099041 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.098850    4865 scope.go:122] "RemoveContainer" containerID="2cc93dbded9dbbd71ea85d8c1bb18370fddc73afad24b1c9c9085d2a9e3e86d4"
Nov 04 02:17:43.100692 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.100668    4865 generic.go:356] "Generic (PLEG): container finished" podID="6f585314-0bb4-48b5-9b4d-80a4e244089c" containerID="7833b88efd58ca5cdfeaf30bc1eaf430db6da8c107bd4a7165317f17e7a860f1" exitCode=0
Nov 04 02:17:43.100845 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.100718    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"
Nov 04 02:17:43.100845 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.100756    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq" event={"ID":"6f585314-0bb4-48b5-9b4d-80a4e244089c","Type":"ContainerDied","Data":"7833b88efd58ca5cdfeaf30bc1eaf430db6da8c107bd4a7165317f17e7a860f1"}
Nov 04 02:17:43.100845 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.100796    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq" event={"ID":"6f585314-0bb4-48b5-9b4d-80a4e244089c","Type":"ContainerDied","Data":"1663c9ca4050b47656a884d77201f33bcc5c03d1a7d10bfd059c25f4e54f0410"}
Nov 04 02:17:43.102996 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.102845    4865 generic.go:356] "Generic (PLEG): container finished" podID="87f5ff3d-b612-4b82-a700-221881b22907" containerID="8d1b814eae74f3709e5a75fd6b63af31e1b508ead98fcb8f78fb1289afa18c41" exitCode=0
Nov 04 02:17:43.102996 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.102951    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26" event={"ID":"87f5ff3d-b612-4b82-a700-221881b22907","Type":"ContainerDied","Data":"8d1b814eae74f3709e5a75fd6b63af31e1b508ead98fcb8f78fb1289afa18c41"}
Nov 04 02:17:43.104019 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.103991    4865 scope.go:122] "RemoveContainer" containerID="8d1b814eae74f3709e5a75fd6b63af31e1b508ead98fcb8f78fb1289afa18c41"
Nov 04 02:17:43.104700 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.104568    4865 generic.go:356] "Generic (PLEG): container finished" podID="cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5" containerID="e1c0aa15d604bcf00bb651c1db3cf63b705184ce6536367912981f0cc63ca7cc" exitCode=0
Nov 04 02:17:43.104700 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.104613    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl" event={"ID":"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5","Type":"ContainerDied","Data":"e1c0aa15d604bcf00bb651c1db3cf63b705184ce6536367912981f0cc63ca7cc"}
Nov 04 02:17:43.104700 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.104638    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl" event={"ID":"cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5","Type":"ContainerDied","Data":"158271a3e30af6138d65d83b91d645ca6cbdbe72f8535ee52a0c79abd20a49e3"}
Nov 04 02:17:43.104700 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.104685    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"
Nov 04 02:17:43.106115 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.106077    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:17:43.106203 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.106160    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:17:43.106262 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.106210    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:17:43.131737 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.131713    4865 scope.go:122] "RemoveContainer" containerID="2cc93dbded9dbbd71ea85d8c1bb18370fddc73afad24b1c9c9085d2a9e3e86d4"
Nov 04 02:17:43.132119 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:17:43.132096    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cc93dbded9dbbd71ea85d8c1bb18370fddc73afad24b1c9c9085d2a9e3e86d4\": container with ID starting with 2cc93dbded9dbbd71ea85d8c1bb18370fddc73afad24b1c9c9085d2a9e3e86d4 not found: ID does not exist" containerID="2cc93dbded9dbbd71ea85d8c1bb18370fddc73afad24b1c9c9085d2a9e3e86d4"
Nov 04 02:17:43.132251 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.132130    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cc93dbded9dbbd71ea85d8c1bb18370fddc73afad24b1c9c9085d2a9e3e86d4"} err="failed to get container status \"2cc93dbded9dbbd71ea85d8c1bb18370fddc73afad24b1c9c9085d2a9e3e86d4\": rpc error: code = NotFound desc = could not find container \"2cc93dbded9dbbd71ea85d8c1bb18370fddc73afad24b1c9c9085d2a9e3e86d4\": container with ID starting with 2cc93dbded9dbbd71ea85d8c1bb18370fddc73afad24b1c9c9085d2a9e3e86d4 not found: ID does not exist"
Nov 04 02:17:43.132251 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.132155    4865 scope.go:122] "RemoveContainer" containerID="7833b88efd58ca5cdfeaf30bc1eaf430db6da8c107bd4a7165317f17e7a860f1"
Nov 04 02:17:43.157382 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.157357    4865 scope.go:122] "RemoveContainer" containerID="7833b88efd58ca5cdfeaf30bc1eaf430db6da8c107bd4a7165317f17e7a860f1"
Nov 04 02:17:43.157856 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:17:43.157821    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7833b88efd58ca5cdfeaf30bc1eaf430db6da8c107bd4a7165317f17e7a860f1\": container with ID starting with 7833b88efd58ca5cdfeaf30bc1eaf430db6da8c107bd4a7165317f17e7a860f1 not found: ID does not exist" containerID="7833b88efd58ca5cdfeaf30bc1eaf430db6da8c107bd4a7165317f17e7a860f1"
Nov 04 02:17:43.157977 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.157867    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7833b88efd58ca5cdfeaf30bc1eaf430db6da8c107bd4a7165317f17e7a860f1"} err="failed to get container status \"7833b88efd58ca5cdfeaf30bc1eaf430db6da8c107bd4a7165317f17e7a860f1\": rpc error: code = NotFound desc = could not find container \"7833b88efd58ca5cdfeaf30bc1eaf430db6da8c107bd4a7165317f17e7a860f1\": container with ID starting with 7833b88efd58ca5cdfeaf30bc1eaf430db6da8c107bd4a7165317f17e7a860f1 not found: ID does not exist"
Nov 04 02:17:43.157977 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.157898    4865 scope.go:122] "RemoveContainer" containerID="e1c0aa15d604bcf00bb651c1db3cf63b705184ce6536367912981f0cc63ca7cc"
Nov 04 02:17:43.173599 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.172473    4865 scope.go:122] "RemoveContainer" containerID="e1c0aa15d604bcf00bb651c1db3cf63b705184ce6536367912981f0cc63ca7cc"
Nov 04 02:17:43.173599 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:17:43.172832    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1c0aa15d604bcf00bb651c1db3cf63b705184ce6536367912981f0cc63ca7cc\": container with ID starting with e1c0aa15d604bcf00bb651c1db3cf63b705184ce6536367912981f0cc63ca7cc not found: ID does not exist" containerID="e1c0aa15d604bcf00bb651c1db3cf63b705184ce6536367912981f0cc63ca7cc"
Nov 04 02:17:43.173599 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.172866    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1c0aa15d604bcf00bb651c1db3cf63b705184ce6536367912981f0cc63ca7cc"} err="failed to get container status \"e1c0aa15d604bcf00bb651c1db3cf63b705184ce6536367912981f0cc63ca7cc\": rpc error: code = NotFound desc = could not find container \"e1c0aa15d604bcf00bb651c1db3cf63b705184ce6536367912981f0cc63ca7cc\": container with ID starting with e1c0aa15d604bcf00bb651c1db3cf63b705184ce6536367912981f0cc63ca7cc not found: ID does not exist"
Nov 04 02:17:43.352783 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.352688    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"]
Nov 04 02:17:43.360872 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.360833    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/distinct-w1-90ad0f-kserve-66fbfd987d-4sjxl"]
Nov 04 02:17:43.593695 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.593657    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"]
Nov 04 02:17:43.596974 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.596586    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/simulator-w1-90ad0f-kserve-55c58ddbc8-2h56v"]
Nov 04 02:17:43.729403 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.729371    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"]
Nov 04 02:17:43.737245 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.737184    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq"]
Nov 04 02:17:43.785606 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:43.785378    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-90ad0f/distinct-2-w1-90ad0f-kserve-79898fdd49-tjsfq" podUID="6f585314-0bb4-48b5-9b4d-80a4e244089c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 02:17:44.112492 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:44.112397    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26" event={"ID":"87f5ff3d-b612-4b82-a700-221881b22907","Type":"ContainerStarted","Data":"791b97814e4673b72b872c553f55f3c3c19109a79a525f7b9b86a3ded1889154"}
Nov 04 02:17:44.289513 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:44.289479    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:17:44.608276 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:44.608239    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:17:44.609624 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:44.609584    4865 patch_prober.go:29] interesting pod/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 04 02:17:44.609760 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:44.609643    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26" podUID="87f5ff3d-b612-4b82-a700-221881b22907" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 04 02:17:44.898335 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:44.898234    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4dfcdafc-3763-472d-b34f-1783a66e838c" path="/var/lib/kubelet/pods/4dfcdafc-3763-472d-b34f-1783a66e838c/volumes"
Nov 04 02:17:44.899033 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:44.899012    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f585314-0bb4-48b5-9b4d-80a4e244089c" path="/var/lib/kubelet/pods/6f585314-0bb4-48b5-9b4d-80a4e244089c/volumes"
Nov 04 02:17:44.899602 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:44.899580    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5" path="/var/lib/kubelet/pods/cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5/volumes"
Nov 04 02:17:45.608390 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:45.608356    4865 patch_prober.go:29] interesting pod/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 04 02:17:45.608855 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:45.608421    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26" podUID="87f5ff3d-b612-4b82-a700-221881b22907" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 04 02:17:46.608429 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:46.608391    4865 patch_prober.go:29] interesting pod/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 04 02:17:46.608814 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:46.608460    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26" podUID="87f5ff3d-b612-4b82-a700-221881b22907" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 04 02:17:47.608912 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:47.608881    4865 patch_prober.go:29] interesting pod/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 04 02:17:47.609326 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:47.608943    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26" podUID="87f5ff3d-b612-4b82-a700-221881b22907" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 04 02:17:48.608019 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:48.607978    4865 patch_prober.go:29] interesting pod/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 04 02:17:48.608217 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:48.608048    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26" podUID="87f5ff3d-b612-4b82-a700-221881b22907" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 04 02:17:49.613189 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:49.613158    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:17:49.614226 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:49.614205    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:17:53.891990 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.891957    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"]
Nov 04 02:17:53.892536 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.892520    4865 cpu_manager.go:414] "containerMap: removing container" podUID="4dfcdafc-3763-472d-b34f-1783a66e838c" containerName="main"
Nov 04 02:17:53.892581 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.892537    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4dfcdafc-3763-472d-b34f-1783a66e838c" containerName="main"
Nov 04 02:17:53.892581 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.892554    4865 cpu_manager.go:414] "containerMap: removing container" podUID="cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5" containerName="main"
Nov 04 02:17:53.892581 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.892559    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5" containerName="main"
Nov 04 02:17:53.892581 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.892576    4865 cpu_manager.go:414] "containerMap: removing container" podUID="6f585314-0bb4-48b5-9b4d-80a4e244089c" containerName="main"
Nov 04 02:17:53.892581 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.892581    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f585314-0bb4-48b5-9b4d-80a4e244089c" containerName="main"
Nov 04 02:17:53.892733 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.892677    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f585314-0bb4-48b5-9b4d-80a4e244089c" containerName="main"
Nov 04 02:17:53.892733 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.892691    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="4dfcdafc-3763-472d-b34f-1783a66e838c" containerName="main"
Nov 04 02:17:53.892733 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.892698    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc04d1bc-8837-4ca4-b6fe-0f292a32d3c5" containerName="main"
Nov 04 02:17:53.895877 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.895859    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:53.942849 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.942823    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"]
Nov 04 02:17:53.946886 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.946865    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:53.981027 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.980995    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aec82805-4138-4238-8b24-44baa8dc2ba4-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:53.981169 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.981033    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/466445d6-ee70-4113-99af-6a9b562b5659-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:53.981314 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.981247    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/466445d6-ee70-4113-99af-6a9b562b5659-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:53.981374 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.981361    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aec82805-4138-4238-8b24-44baa8dc2ba4-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:53.981465 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.981441    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aec82805-4138-4238-8b24-44baa8dc2ba4-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:53.981555 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.981536    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aec82805-4138-4238-8b24-44baa8dc2ba4-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:53.981614 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.981582    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/466445d6-ee70-4113-99af-6a9b562b5659-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:53.981673 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.981651    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aec82805-4138-4238-8b24-44baa8dc2ba4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:53.981730 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.981709    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aec82805-4138-4238-8b24-44baa8dc2ba4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:53.981838 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.981816    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/466445d6-ee70-4113-99af-6a9b562b5659-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:53.981906 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.981882    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/466445d6-ee70-4113-99af-6a9b562b5659-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:53.981975 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.981961    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/466445d6-ee70-4113-99af-6a9b562b5659-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:53.982021 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.981987    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec82805-4138-4238-8b24-44baa8dc2ba4-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:53.982071 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.982034    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29vnw\" (UniqueName: \"kubernetes.io/projected/466445d6-ee70-4113-99af-6a9b562b5659-kube-api-access-29vnw\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:53.982071 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.982064    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec82805-4138-4238-8b24-44baa8dc2ba4-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:53.982200 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.982186    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/466445d6-ee70-4113-99af-6a9b562b5659-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:53.982256 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.982207    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/466445d6-ee70-4113-99af-6a9b562b5659-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:53.982331 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:53.982257    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29d5c\" (UniqueName: \"kubernetes.io/projected/aec82805-4138-4238-8b24-44baa8dc2ba4-kube-api-access-29d5c\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.041893 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.041859    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"]
Nov 04 02:17:54.083867 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.083838    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aec82805-4138-4238-8b24-44baa8dc2ba4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.084084 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.083881    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/466445d6-ee70-4113-99af-6a9b562b5659-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.084084 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.083914    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/466445d6-ee70-4113-99af-6a9b562b5659-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.084084 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.083935    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/466445d6-ee70-4113-99af-6a9b562b5659-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.084084 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.083957    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec82805-4138-4238-8b24-44baa8dc2ba4-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.084084 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.083992    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29vnw\" (UniqueName: \"kubernetes.io/projected/466445d6-ee70-4113-99af-6a9b562b5659-kube-api-access-29vnw\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.084379 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084249    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec82805-4138-4238-8b24-44baa8dc2ba4-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.084379 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084324    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/466445d6-ee70-4113-99af-6a9b562b5659-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.084379 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084351    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/466445d6-ee70-4113-99af-6a9b562b5659-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.084522 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084380    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29d5c\" (UniqueName: \"kubernetes.io/projected/aec82805-4138-4238-8b24-44baa8dc2ba4-kube-api-access-29d5c\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.084522 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084414    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/466445d6-ee70-4113-99af-6a9b562b5659-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.084522 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084441    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/466445d6-ee70-4113-99af-6a9b562b5659-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.084522 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084452    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aec82805-4138-4238-8b24-44baa8dc2ba4-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.084522 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084498    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/466445d6-ee70-4113-99af-6a9b562b5659-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.084772 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084570    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/466445d6-ee70-4113-99af-6a9b562b5659-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.084772 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084596    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec82805-4138-4238-8b24-44baa8dc2ba4-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.084772 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084644    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aec82805-4138-4238-8b24-44baa8dc2ba4-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.084772 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084681    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aec82805-4138-4238-8b24-44baa8dc2ba4-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.084772 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084724    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aec82805-4138-4238-8b24-44baa8dc2ba4-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.084772 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084762    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/466445d6-ee70-4113-99af-6a9b562b5659-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.085092 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084798    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aec82805-4138-4238-8b24-44baa8dc2ba4-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.085092 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084831    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/466445d6-ee70-4113-99af-6a9b562b5659-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.085092 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084835    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aec82805-4138-4238-8b24-44baa8dc2ba4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.085092 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084897    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec82805-4138-4238-8b24-44baa8dc2ba4-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.085092 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.084976    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/466445d6-ee70-4113-99af-6a9b562b5659-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.085461 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.085429    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aec82805-4138-4238-8b24-44baa8dc2ba4-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.085631 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.085610    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/466445d6-ee70-4113-99af-6a9b562b5659-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.085818 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.085789    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aec82805-4138-4238-8b24-44baa8dc2ba4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.087642 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.087615    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aec82805-4138-4238-8b24-44baa8dc2ba4-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.087742 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.087646    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aec82805-4138-4238-8b24-44baa8dc2ba4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.087806 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.087781    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/466445d6-ee70-4113-99af-6a9b562b5659-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.087844 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.087808    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/466445d6-ee70-4113-99af-6a9b562b5659-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.115328 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.115259    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"]
Nov 04 02:17:54.146763 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.146689    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29vnw\" (UniqueName: \"kubernetes.io/projected/466445d6-ee70-4113-99af-6a9b562b5659-kube-api-access-29vnw\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.146940 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.146914    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29d5c\" (UniqueName: \"kubernetes.io/projected/aec82805-4138-4238-8b24-44baa8dc2ba4-kube-api-access-29d5c\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.147164 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.147141    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aec82805-4138-4238-8b24-44baa8dc2ba4-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p\" (UID: \"aec82805-4138-4238-8b24-44baa8dc2ba4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.147412 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.147390    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/466445d6-ee70-4113-99af-6a9b562b5659-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b9chv\" (UID: \"466445d6-ee70-4113-99af-6a9b562b5659\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.209212 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.209181    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:54.259787 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.259737    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:54.637508 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.637481    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"]
Nov 04 02:17:54.638046 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:17:54.638025    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod466445d6_ee70_4113_99af_6a9b562b5659.slice/crio-94da73e8d208d5f7bcbf8810a42d3ff4fc5e6860afc913eb82436481ee040d04 WatchSource:0}: Error finding container 94da73e8d208d5f7bcbf8810a42d3ff4fc5e6860afc913eb82436481ee040d04: Status 404 returned error can't find the container with id 94da73e8d208d5f7bcbf8810a42d3ff4fc5e6860afc913eb82436481ee040d04
Nov 04 02:17:54.640435 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.640407    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:17:54.640503 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.640465    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:17:54.640503 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.640492    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:17:54.730166 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:17:54.730109    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec82805_4138_4238_8b24_44baa8dc2ba4.slice/crio-f79a09b1f3b7150894491cd11be8b919e906d6d2ed338c7237e31de0e37c30b9 WatchSource:0}: Error finding container f79a09b1f3b7150894491cd11be8b919e906d6d2ed338c7237e31de0e37c30b9: Status 404 returned error can't find the container with id f79a09b1f3b7150894491cd11be8b919e906d6d2ed338c7237e31de0e37c30b9
Nov 04 02:17:54.735856 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.734699    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:17:54.735856 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.734794    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:17:54.735856 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.734835    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:17:54.735856 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:54.735694    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"]
Nov 04 02:17:55.236940 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:55.236900    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p" event={"ID":"aec82805-4138-4238-8b24-44baa8dc2ba4","Type":"ContainerStarted","Data":"14e95d739cea7e313ca6f646876280581b734b76f639b93a7a974ce58833df54"}
Nov 04 02:17:55.237418 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:55.236984    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p" event={"ID":"aec82805-4138-4238-8b24-44baa8dc2ba4","Type":"ContainerStarted","Data":"f79a09b1f3b7150894491cd11be8b919e906d6d2ed338c7237e31de0e37c30b9"}
Nov 04 02:17:55.238914 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:55.238885    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv" event={"ID":"466445d6-ee70-4113-99af-6a9b562b5659","Type":"ContainerStarted","Data":"7e2e69ca30fa27279e2657e8a711596809d0b74a73dcb0501c445521de7cf00d"}
Nov 04 02:17:55.239045 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:55.238940    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv" event={"ID":"466445d6-ee70-4113-99af-6a9b562b5659","Type":"ContainerStarted","Data":"94da73e8d208d5f7bcbf8810a42d3ff4fc5e6860afc913eb82436481ee040d04"}
Nov 04 02:17:55.260340 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:55.260304    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:17:55.262342 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:55.262313    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:17:55.262440 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:55.262373    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p" podUID="aec82805-4138-4238-8b24-44baa8dc2ba4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 02:17:55.361574 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:55.361516    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p" podStartSLOduration=2.361501859 podStartE2EDuration="2.361501859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:17:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:17:55.361454349 +0000 UTC m=+1338.588241019" watchObservedRunningTime="2026-11-04 02:17:55.361501859 +0000 UTC m=+1338.588288527"
Nov 04 02:17:55.488437 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:55.488322    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv" podStartSLOduration=2.488305774 podStartE2EDuration="2.488305774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:17:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:17:55.485006214 +0000 UTC m=+1338.711792882" watchObservedRunningTime="2026-11-04 02:17:55.488305774 +0000 UTC m=+1338.715092443"
Nov 04 02:17:56.210374 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:56.210337    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:17:56.211991 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:56.211960    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:17:56.212101 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:56.212013    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv" podUID="466445d6-ee70-4113-99af-6a9b562b5659" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:17:56.276306 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:56.275715    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:17:56.276306 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:56.275778    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p" podUID="aec82805-4138-4238-8b24-44baa8dc2ba4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 02:17:57.209860 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:57.209826    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:17:57.210054 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:57.209936    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv" podUID="466445d6-ee70-4113-99af-6a9b562b5659" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:17:57.263306 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:57.260759    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:17:57.263306 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:57.260835    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p" podUID="aec82805-4138-4238-8b24-44baa8dc2ba4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 02:17:58.210067 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:58.210035    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:17:58.210499 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:58.210473    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv" podUID="466445d6-ee70-4113-99af-6a9b562b5659" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:17:58.266707 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:58.266652    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:17:58.266887 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:58.266718    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p" podUID="aec82805-4138-4238-8b24-44baa8dc2ba4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 02:17:58.413348 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:17:58.413310    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:17:58.511637 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:58.511555    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:17:59.210366 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:59.210321    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:17:59.210882 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:59.210407    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv" podUID="466445d6-ee70-4113-99af-6a9b562b5659" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:17:59.260685 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:59.260646    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:17:59.260940 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:17:59.260913    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p" podUID="aec82805-4138-4238-8b24-44baa8dc2ba4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 02:18:00.209929 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:00.209884    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:18:00.210228 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:00.210178    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv" podUID="466445d6-ee70-4113-99af-6a9b562b5659" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:18:00.260784 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:00.260742    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:18:00.261453 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:00.261419    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p" podUID="aec82805-4138-4238-8b24-44baa8dc2ba4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 02:18:01.119028 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.115375    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"]
Nov 04 02:18:01.134583 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.134548    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:18:01.192304 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.192098    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"]
Nov 04 02:18:01.210697 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.210663    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:18:01.217413 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.217371    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv" podUID="466445d6-ee70-4113-99af-6a9b562b5659" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:18:01.235713 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.235676    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-3f76d0-serving-cert\""
Nov 04 02:18:01.263671 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.263638    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:18:01.289429 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.289384    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p" podUID="aec82805-4138-4238-8b24-44baa8dc2ba4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 02:18:01.320138 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.320093    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r\" (UID: \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:18:01.467652 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.440536    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-maas-api-tls\") pod \"maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r\" (UID: \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:18:01.467652 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.440736    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8kb7\" (UniqueName: \"kubernetes.io/projected/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-kube-api-access-d8kb7\") pod \"maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r\" (UID: \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:18:01.467652 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.440867    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-metrics-certs\") pod \"maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r\" (UID: \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:18:01.669574 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.669534    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-maas-api-tls\") pod \"maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r\" (UID: \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:18:01.670366 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.670337    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8kb7\" (UniqueName: \"kubernetes.io/projected/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-kube-api-access-d8kb7\") pod \"maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r\" (UID: \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:18:01.670650 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.670631    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-metrics-certs\") pod \"maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r\" (UID: \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:18:01.670876 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.670856    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r\" (UID: \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:18:01.672461 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.672439    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r\" (UID: \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:18:01.692011 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.691921    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-metrics-certs\") pod \"maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r\" (UID: \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:18:01.697138 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.697111    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-maas-api-tls\") pod \"maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r\" (UID: \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:18:01.735141 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.735064    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8kb7\" (UniqueName: \"kubernetes.io/projected/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-kube-api-access-d8kb7\") pod \"maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r\" (UID: \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:18:01.785446 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:01.785407    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:18:02.173035 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:18:02.172998    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dc8d5b0_f8df_405b_a02d_3ca353f7ecbb.slice/crio-be23f4836cf1a0c0291e99ad95a8b62c64538d23be6279478180ffa476f250f2 WatchSource:0}: Error finding container be23f4836cf1a0c0291e99ad95a8b62c64538d23be6279478180ffa476f250f2: Status 404 returned error can't find the container with id be23f4836cf1a0c0291e99ad95a8b62c64538d23be6279478180ffa476f250f2
Nov 04 02:18:02.173462 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:02.173436    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"]
Nov 04 02:18:02.211216 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:02.211188    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:18:02.211388 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:02.211243    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv" podUID="466445d6-ee70-4113-99af-6a9b562b5659" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:18:02.261176 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:02.261136    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:18:02.261358 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:02.261198    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p" podUID="aec82805-4138-4238-8b24-44baa8dc2ba4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 02:18:02.680778 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:02.680733    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r" event={"ID":"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb","Type":"ContainerStarted","Data":"be23f4836cf1a0c0291e99ad95a8b62c64538d23be6279478180ffa476f250f2"}
Nov 04 02:18:03.209673 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:03.209642    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:18:03.209820 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:03.209704    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv" podUID="466445d6-ee70-4113-99af-6a9b562b5659" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 02:18:03.260096 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:03.260056    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:18:03.260269 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:03.260119    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p" podUID="aec82805-4138-4238-8b24-44baa8dc2ba4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 02:18:03.692958 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:03.692914    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r" event={"ID":"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb","Type":"ContainerStarted","Data":"ab64282eca5aeb206849dc6971fb26abf3ff4088e38b6cd46ce8c515ccc3f70b"}
Nov 04 02:18:03.693558 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:03.693153    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:18:03.817554 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:03.817472    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r" podStartSLOduration=1.873954788 podStartE2EDuration="2.817455556s" totalImagesPullingTime="943.500768ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:18:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:18:03.813200003 +0000 UTC m=+1347.039986673" watchObservedRunningTime="2026-11-04 02:18:03.817455556 +0000 UTC m=+1347.044242226"
Nov 04 02:18:04.864677 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:04.864637    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:18:04.865612 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:04.865587    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:18:04.866616 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:04.866591    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv"
Nov 04 02:18:04.884568 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:04.884537    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:18:04.884988 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:04.884970    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:18:04.886841 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:04.886813    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p"
Nov 04 02:18:05.774534 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:05.774491    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"]
Nov 04 02:18:05.774784 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:05.774758    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt" podUID="a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1" containerName="maas-api" containerID="cri-o://bd5b7e1f5ed7b42c8a667e757fc11d81cfcc6d6ab9e1564b86c83f4f0c4cf945" gracePeriod=30
Nov 04 02:18:06.131096 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.131071    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:18:06.199883 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.199838    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-metrics-certs\") pod \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\" (UID: \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\") "
Nov 04 02:18:06.200382 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.200352    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-maas-api-tls\") pod \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\" (UID: \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\") "
Nov 04 02:18:06.200542 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.200525    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-openshift-service-ca\") pod \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\" (UID: \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\") "
Nov 04 02:18:06.200665 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.200651    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-kube-api-access-hsrvh\" (UniqueName: \"kubernetes.io/projected/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-kube-api-access-hsrvh\") pod \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\" (UID: \"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1\") "
Nov 04 02:18:06.210020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.209671    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-openshift-service-ca" pod "a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1" (UID: "a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:18:06.218570 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.218535    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-maas-api-tls" pod "a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1" (UID: "a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:18:06.218812 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.218776    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-kube-api-access-hsrvh" pod "a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1" (UID: "a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1"). InnerVolumeSpecName "kube-api-access-hsrvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:18:06.219441 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.219399    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-metrics-certs" pod "a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1" (UID: "a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:18:06.302728 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.302694    4865 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-maas-api-tls\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:06.302728 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.302726    4865 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-openshift-service-ca\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:06.302936 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.302736    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsrvh\" (UniqueName: \"kubernetes.io/projected/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-kube-api-access-hsrvh\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:06.302936 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.302747    4865 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1-metrics-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:06.770864 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.770828    4865 generic.go:356] "Generic (PLEG): container finished" podID="a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1" containerID="bd5b7e1f5ed7b42c8a667e757fc11d81cfcc6d6ab9e1564b86c83f4f0c4cf945" exitCode=0
Nov 04 02:18:06.771958 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.771935    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"
Nov 04 02:18:06.816484 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.816448    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt" event={"ID":"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1","Type":"ContainerDied","Data":"bd5b7e1f5ed7b42c8a667e757fc11d81cfcc6d6ab9e1564b86c83f4f0c4cf945"}
Nov 04 02:18:06.816729 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.816702    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt" event={"ID":"a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1","Type":"ContainerDied","Data":"5853ffa22200d906f0d9cf5a141fd5b074ebb2a549296ced9a412e4b6e1aba10"}
Nov 04 02:18:06.816879 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.816850    4865 scope.go:122] "RemoveContainer" containerID="bd5b7e1f5ed7b42c8a667e757fc11d81cfcc6d6ab9e1564b86c83f4f0c4cf945"
Nov 04 02:18:06.898468 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.898440    4865 scope.go:122] "RemoveContainer" containerID="bd5b7e1f5ed7b42c8a667e757fc11d81cfcc6d6ab9e1564b86c83f4f0c4cf945"
Nov 04 02:18:06.899108 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:18:06.899081    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd5b7e1f5ed7b42c8a667e757fc11d81cfcc6d6ab9e1564b86c83f4f0c4cf945\": container with ID starting with bd5b7e1f5ed7b42c8a667e757fc11d81cfcc6d6ab9e1564b86c83f4f0c4cf945 not found: ID does not exist" containerID="bd5b7e1f5ed7b42c8a667e757fc11d81cfcc6d6ab9e1564b86c83f4f0c4cf945"
Nov 04 02:18:06.899269 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.899249    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd5b7e1f5ed7b42c8a667e757fc11d81cfcc6d6ab9e1564b86c83f4f0c4cf945"} err="failed to get container status \"bd5b7e1f5ed7b42c8a667e757fc11d81cfcc6d6ab9e1564b86c83f4f0c4cf945\": rpc error: code = NotFound desc = could not find container \"bd5b7e1f5ed7b42c8a667e757fc11d81cfcc6d6ab9e1564b86c83f4f0c4cf945\": container with ID starting with bd5b7e1f5ed7b42c8a667e757fc11d81cfcc6d6ab9e1564b86c83f4f0c4cf945 not found: ID does not exist"
Nov 04 02:18:06.991436 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:06.991405    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"]
Nov 04 02:18:07.043359 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:07.043266    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-90ad0f-66459bc677-xv5pt"]
Nov 04 02:18:08.906319 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:08.905935    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1" path="/var/lib/kubelet/pods/a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1/volumes"
Nov 04 02:18:09.034471 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.034435    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"]
Nov 04 02:18:09.035025 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.035011    4865 cpu_manager.go:414] "containerMap: removing container" podUID="a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1" containerName="maas-api"
Nov 04 02:18:09.035077 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.035026    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1" containerName="maas-api"
Nov 04 02:18:09.035134 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.035125    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="a82e2dda-81f8-4c69-8f84-6a2a6fafb9a1" containerName="maas-api"
Nov 04 02:18:09.039895 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.039873    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.124801 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.124759    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"]
Nov 04 02:18:09.152644 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.152608    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-3d082d-openshift-default-dockercfg-k6xhs\""
Nov 04 02:18:09.200715 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.200682    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-workload-certs\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.201024 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.201007    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/707f17bc-7327-4cd7-9796-497331e55d9c-istiod-ca-cert\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.201182 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.201159    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66zfq\" (UniqueName: \"kubernetes.io/projected/707f17bc-7327-4cd7-9796-497331e55d9c-kube-api-access-66zfq\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.201321 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.201304    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-workload-socket\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.201525 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.201509    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-istio-envoy\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.201639 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.201628    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/707f17bc-7327-4cd7-9796-497331e55d9c-istio-token\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.201805 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.201786    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/707f17bc-7327-4cd7-9796-497331e55d9c-istio-podinfo\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.201935 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.201907    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-istio-data\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.202008 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.201953    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-credential-socket\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.329003 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.328490    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/707f17bc-7327-4cd7-9796-497331e55d9c-istiod-ca-cert\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.329003 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.328603    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66zfq\" (UniqueName: \"kubernetes.io/projected/707f17bc-7327-4cd7-9796-497331e55d9c-kube-api-access-66zfq\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.329003 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.328649    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-workload-socket\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.329003 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.328754    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-istio-envoy\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.329003 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.328784    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/707f17bc-7327-4cd7-9796-497331e55d9c-istio-token\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.329003 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.328828    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/707f17bc-7327-4cd7-9796-497331e55d9c-istio-podinfo\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.329003 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.328906    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-istio-data\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.329003 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.328939    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-credential-socket\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.331549 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.329567    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/707f17bc-7327-4cd7-9796-497331e55d9c-istiod-ca-cert\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.331549 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.329784    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-workload-certs\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.331549 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.329797    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-workload-socket\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.331549 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.330076    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-istio-data\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.331549 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.330348    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-credential-socket\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.331549 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.330612    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-workload-certs\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.345494 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.344129    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/707f17bc-7327-4cd7-9796-497331e55d9c-istio-podinfo\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.439856 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.439724    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/707f17bc-7327-4cd7-9796-497331e55d9c-istio-token\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.439856 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.439746    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66zfq\" (UniqueName: \"kubernetes.io/projected/707f17bc-7327-4cd7-9796-497331e55d9c-kube-api-access-66zfq\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.449442 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.449386    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-istio-envoy\") pod \"e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") " pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.663041 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.662948    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:09.709918 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:09.709885    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:18:10.096255 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:10.096223    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"]
Nov 04 02:18:10.096673 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:18:10.096487    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod707f17bc_7327_4cd7_9796_497331e55d9c.slice/crio-8b50128d9c8497d1c33975a1f0efc2eeefe18d961650a1c6abe1e7923c9b7471 WatchSource:0}: Error finding container 8b50128d9c8497d1c33975a1f0efc2eeefe18d961650a1c6abe1e7923c9b7471: Status 404 returned error can't find the container with id 8b50128d9c8497d1c33975a1f0efc2eeefe18d961650a1c6abe1e7923c9b7471
Nov 04 02:18:10.099085 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:10.099056    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:18:10.099145 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:10.099116    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:18:10.099180 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:10.099146    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:18:10.415683 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:10.415591    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"]
Nov 04 02:18:10.416599 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:10.416109    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j" podUID="1c786c4c-0ace-4ac4-aad0-3984cb251c72" containerName="istio-proxy" containerID="cri-o://ebea8f67a82330104fc86273028fa40cc71b103afa1f169eb9a0c2fc2a93e0a0" gracePeriod=30
Nov 04 02:18:10.423006 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:10.422984    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"]
Nov 04 02:18:10.423424 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:10.423361    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh" podUID="3266fdea-562e-4e27-8b95-c1157b563921" containerName="istio-proxy" containerID="cri-o://a76eea58e2114802ff0e889b1465b5e2c68ad143eb4089836c3ed97389937a4b" gracePeriod=30
Nov 04 02:18:10.800958 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:10.800921    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd" event={"ID":"707f17bc-7327-4cd7-9796-497331e55d9c","Type":"ContainerStarted","Data":"d1e597d977231f66f057dc7f061c3ef64c330640ccbab84515f07b9b7b868170"}
Nov 04 02:18:10.801140 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:10.800983    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd" event={"ID":"707f17bc-7327-4cd7-9796-497331e55d9c","Type":"ContainerStarted","Data":"8b50128d9c8497d1c33975a1f0efc2eeefe18d961650a1c6abe1e7923c9b7471"}
Nov 04 02:18:10.925048 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:10.924986    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd" podStartSLOduration=2.924974152 podStartE2EDuration="2.924974152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:18:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:18:10.922573729 +0000 UTC m=+1354.149360398" watchObservedRunningTime="2026-11-04 02:18:10.924974152 +0000 UTC m=+1354.151760820"
Nov 04 02:18:11.663173 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:11.663132    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:11.668965 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:11.668934    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:11.806980 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:11.806941    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:11.807971 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:11.807948    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:18:15.718162 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.718135    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:18:15.743580 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.743553    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:18:15.821014 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.820972    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-podinfo\") pod \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") "
Nov 04 02:18:15.821187 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821049    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-workload-socket\") pod \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") "
Nov 04 02:18:15.821257 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821223    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3266fdea-562e-4e27-8b95-c1157b563921-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3266fdea-562e-4e27-8b95-c1157b563921-istiod-ca-cert\") pod \"3266fdea-562e-4e27-8b95-c1157b563921\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") "
Nov 04 02:18:15.821382 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821311    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-workload-certs\") pod \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") "
Nov 04 02:18:15.821382 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821365    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-workload-socket" pod "1c786c4c-0ace-4ac4-aad0-3984cb251c72" (UID: "1c786c4c-0ace-4ac4-aad0-3984cb251c72"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:15.821500 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821368    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-token\" (UniqueName: \"kubernetes.io/projected/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-token\") pod \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") "
Nov 04 02:18:15.821500 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821422    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-credential-socket\") pod \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") "
Nov 04 02:18:15.821500 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821481    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-istio-envoy\") pod \"3266fdea-562e-4e27-8b95-c1157b563921\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") "
Nov 04 02:18:15.821652 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821509    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-workload-socket\") pod \"3266fdea-562e-4e27-8b95-c1157b563921\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") "
Nov 04 02:18:15.821652 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821539    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3266fdea-562e-4e27-8b95-c1157b563921-istio-token\" (UniqueName: \"kubernetes.io/projected/3266fdea-562e-4e27-8b95-c1157b563921-istio-token\") pod \"3266fdea-562e-4e27-8b95-c1157b563921\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") "
Nov 04 02:18:15.821652 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821569    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-workload-certs" pod "1c786c4c-0ace-4ac4-aad0-3984cb251c72" (UID: "1c786c4c-0ace-4ac4-aad0-3984cb251c72"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:15.821652 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821576    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c786c4c-0ace-4ac4-aad0-3984cb251c72-kube-api-access-r7xkt\" (UniqueName: \"kubernetes.io/projected/1c786c4c-0ace-4ac4-aad0-3984cb251c72-kube-api-access-r7xkt\") pod \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") "
Nov 04 02:18:15.821652 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821611    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3266fdea-562e-4e27-8b95-c1157b563921-istiod-ca-cert" pod "3266fdea-562e-4e27-8b95-c1157b563921" (UID: "3266fdea-562e-4e27-8b95-c1157b563921"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:18:15.821652 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821634    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istiod-ca-cert\") pod \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") "
Nov 04 02:18:15.821978 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821676    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3266fdea-562e-4e27-8b95-c1157b563921-kube-api-access-rgj7x\" (UniqueName: \"kubernetes.io/projected/3266fdea-562e-4e27-8b95-c1157b563921-kube-api-access-rgj7x\") pod \"3266fdea-562e-4e27-8b95-c1157b563921\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") "
Nov 04 02:18:15.821978 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821710    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3266fdea-562e-4e27-8b95-c1157b563921-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3266fdea-562e-4e27-8b95-c1157b563921-istio-podinfo\") pod \"3266fdea-562e-4e27-8b95-c1157b563921\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") "
Nov 04 02:18:15.821978 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821799    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-istio-data\") pod \"3266fdea-562e-4e27-8b95-c1157b563921\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") "
Nov 04 02:18:15.821978 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821840    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-workload-certs\") pod \"3266fdea-562e-4e27-8b95-c1157b563921\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") "
Nov 04 02:18:15.821978 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.821933    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-credential-socket\") pod \"3266fdea-562e-4e27-8b95-c1157b563921\" (UID: \"3266fdea-562e-4e27-8b95-c1157b563921\") "
Nov 04 02:18:15.822239 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.822018    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-envoy\") pod \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") "
Nov 04 02:18:15.822239 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.822109    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-data\") pod \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\" (UID: \"1c786c4c-0ace-4ac4-aad0-3984cb251c72\") "
Nov 04 02:18:15.822707 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.822634    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-credential-socket" pod "1c786c4c-0ace-4ac4-aad0-3984cb251c72" (UID: "1c786c4c-0ace-4ac4-aad0-3984cb251c72"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:15.823915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.822807    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-workload-socket" pod "3266fdea-562e-4e27-8b95-c1157b563921" (UID: "3266fdea-562e-4e27-8b95-c1157b563921"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:15.823915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.822960    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-credential-socket" pod "3266fdea-562e-4e27-8b95-c1157b563921" (UID: "3266fdea-562e-4e27-8b95-c1157b563921"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:15.823915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.823028    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-workload-certs" pod "3266fdea-562e-4e27-8b95-c1157b563921" (UID: "3266fdea-562e-4e27-8b95-c1157b563921"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:15.823915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.823364    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-data" pod "1c786c4c-0ace-4ac4-aad0-3984cb251c72" (UID: "1c786c4c-0ace-4ac4-aad0-3984cb251c72"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:15.823915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.823450    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istiod-ca-cert" pod "1c786c4c-0ace-4ac4-aad0-3984cb251c72" (UID: "1c786c4c-0ace-4ac4-aad0-3984cb251c72"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:18:15.823915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.823645    4865 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-workload-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.823915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.823665    4865 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istiod-ca-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.823915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.823680    4865 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-workload-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.823915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.823694    4865 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-credential-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.823915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.823708    4865 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-data\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.823915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.823721    4865 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-workload-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.823915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.823734    4865 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3266fdea-562e-4e27-8b95-c1157b563921-istiod-ca-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.823915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.823748    4865 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-workload-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.823915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.823761    4865 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-credential-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.823915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.823832    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-podinfo" pod "1c786c4c-0ace-4ac4-aad0-3984cb251c72" (UID: "1c786c4c-0ace-4ac4-aad0-3984cb251c72"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:18:15.823915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.823895    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-istio-data" pod "3266fdea-562e-4e27-8b95-c1157b563921" (UID: "3266fdea-562e-4e27-8b95-c1157b563921"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:15.825013 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.824402    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-token" pod "1c786c4c-0ace-4ac4-aad0-3984cb251c72" (UID: "1c786c4c-0ace-4ac4-aad0-3984cb251c72"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:18:15.825013 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.824434    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c786c4c-0ace-4ac4-aad0-3984cb251c72-kube-api-access-r7xkt" pod "1c786c4c-0ace-4ac4-aad0-3984cb251c72" (UID: "1c786c4c-0ace-4ac4-aad0-3984cb251c72"). InnerVolumeSpecName "kube-api-access-r7xkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:18:15.825013 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.824763    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-istio-envoy" pod "3266fdea-562e-4e27-8b95-c1157b563921" (UID: "3266fdea-562e-4e27-8b95-c1157b563921"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:15.825308 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.825264    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-envoy" pod "1c786c4c-0ace-4ac4-aad0-3984cb251c72" (UID: "1c786c4c-0ace-4ac4-aad0-3984cb251c72"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:15.825711 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.825691    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3266fdea-562e-4e27-8b95-c1157b563921-kube-api-access-rgj7x" pod "3266fdea-562e-4e27-8b95-c1157b563921" (UID: "3266fdea-562e-4e27-8b95-c1157b563921"). InnerVolumeSpecName "kube-api-access-rgj7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:18:15.825833 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.825819    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3266fdea-562e-4e27-8b95-c1157b563921-istio-token" pod "3266fdea-562e-4e27-8b95-c1157b563921" (UID: "3266fdea-562e-4e27-8b95-c1157b563921"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:18:15.826427 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.826414    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3266fdea-562e-4e27-8b95-c1157b563921-istio-podinfo" pod "3266fdea-562e-4e27-8b95-c1157b563921" (UID: "3266fdea-562e-4e27-8b95-c1157b563921"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:18:15.833883 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.833860    4865 generic.go:356] "Generic (PLEG): container finished" podID="1c786c4c-0ace-4ac4-aad0-3984cb251c72" containerID="ebea8f67a82330104fc86273028fa40cc71b103afa1f169eb9a0c2fc2a93e0a0" exitCode=0
Nov 04 02:18:15.833985 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.833912    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"
Nov 04 02:18:15.833985 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.833940    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j" event={"ID":"1c786c4c-0ace-4ac4-aad0-3984cb251c72","Type":"ContainerDied","Data":"ebea8f67a82330104fc86273028fa40cc71b103afa1f169eb9a0c2fc2a93e0a0"}
Nov 04 02:18:15.834098 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.833993    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j" event={"ID":"1c786c4c-0ace-4ac4-aad0-3984cb251c72","Type":"ContainerDied","Data":"5bf5d1cc722cca2c94eb4c8bc850386fb80e1c15edb17a949f6c9f092f0f34b1"}
Nov 04 02:18:15.834098 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.834016    4865 scope.go:122] "RemoveContainer" containerID="ebea8f67a82330104fc86273028fa40cc71b103afa1f169eb9a0c2fc2a93e0a0"
Nov 04 02:18:15.835599 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.835574    4865 generic.go:356] "Generic (PLEG): container finished" podID="3266fdea-562e-4e27-8b95-c1157b563921" containerID="a76eea58e2114802ff0e889b1465b5e2c68ad143eb4089836c3ed97389937a4b" exitCode=0
Nov 04 02:18:15.835703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.835611    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"
Nov 04 02:18:15.835703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.835647    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh" event={"ID":"3266fdea-562e-4e27-8b95-c1157b563921","Type":"ContainerDied","Data":"a76eea58e2114802ff0e889b1465b5e2c68ad143eb4089836c3ed97389937a4b"}
Nov 04 02:18:15.835703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.835689    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh" event={"ID":"3266fdea-562e-4e27-8b95-c1157b563921","Type":"ContainerDied","Data":"0b310f3621a5b04e7f2745f37985537b0bcc8d5d053574cb065ff27db7ed2dba"}
Nov 04 02:18:15.845094 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.845073    4865 scope.go:122] "RemoveContainer" containerID="16fcb3dcb47a5e67eb8bc90ed64fe93d83d052f756ec655b19fcbf263fc52075"
Nov 04 02:18:15.868194 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.868165    4865 scope.go:122] "RemoveContainer" containerID="ebea8f67a82330104fc86273028fa40cc71b103afa1f169eb9a0c2fc2a93e0a0"
Nov 04 02:18:15.868821 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:18:15.868791    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebea8f67a82330104fc86273028fa40cc71b103afa1f169eb9a0c2fc2a93e0a0\": container with ID starting with ebea8f67a82330104fc86273028fa40cc71b103afa1f169eb9a0c2fc2a93e0a0 not found: ID does not exist" containerID="ebea8f67a82330104fc86273028fa40cc71b103afa1f169eb9a0c2fc2a93e0a0"
Nov 04 02:18:15.868917 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.868837    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebea8f67a82330104fc86273028fa40cc71b103afa1f169eb9a0c2fc2a93e0a0"} err="failed to get container status \"ebea8f67a82330104fc86273028fa40cc71b103afa1f169eb9a0c2fc2a93e0a0\": rpc error: code = NotFound desc = could not find container \"ebea8f67a82330104fc86273028fa40cc71b103afa1f169eb9a0c2fc2a93e0a0\": container with ID starting with ebea8f67a82330104fc86273028fa40cc71b103afa1f169eb9a0c2fc2a93e0a0 not found: ID does not exist"
Nov 04 02:18:15.868917 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.868862    4865 scope.go:122] "RemoveContainer" containerID="16fcb3dcb47a5e67eb8bc90ed64fe93d83d052f756ec655b19fcbf263fc52075"
Nov 04 02:18:15.869262 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:18:15.869236    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16fcb3dcb47a5e67eb8bc90ed64fe93d83d052f756ec655b19fcbf263fc52075\": container with ID starting with 16fcb3dcb47a5e67eb8bc90ed64fe93d83d052f756ec655b19fcbf263fc52075 not found: ID does not exist" containerID="16fcb3dcb47a5e67eb8bc90ed64fe93d83d052f756ec655b19fcbf263fc52075"
Nov 04 02:18:15.869410 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.869276    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16fcb3dcb47a5e67eb8bc90ed64fe93d83d052f756ec655b19fcbf263fc52075"} err="failed to get container status \"16fcb3dcb47a5e67eb8bc90ed64fe93d83d052f756ec655b19fcbf263fc52075\": rpc error: code = NotFound desc = could not find container \"16fcb3dcb47a5e67eb8bc90ed64fe93d83d052f756ec655b19fcbf263fc52075\": container with ID starting with 16fcb3dcb47a5e67eb8bc90ed64fe93d83d052f756ec655b19fcbf263fc52075 not found: ID does not exist"
Nov 04 02:18:15.869410 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.869334    4865 scope.go:122] "RemoveContainer" containerID="a76eea58e2114802ff0e889b1465b5e2c68ad143eb4089836c3ed97389937a4b"
Nov 04 02:18:15.879774 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.879738    4865 scope.go:122] "RemoveContainer" containerID="a76eea58e2114802ff0e889b1465b5e2c68ad143eb4089836c3ed97389937a4b"
Nov 04 02:18:15.880083 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:18:15.880062    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a76eea58e2114802ff0e889b1465b5e2c68ad143eb4089836c3ed97389937a4b\": container with ID starting with a76eea58e2114802ff0e889b1465b5e2c68ad143eb4089836c3ed97389937a4b not found: ID does not exist" containerID="a76eea58e2114802ff0e889b1465b5e2c68ad143eb4089836c3ed97389937a4b"
Nov 04 02:18:15.880149 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.880090    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a76eea58e2114802ff0e889b1465b5e2c68ad143eb4089836c3ed97389937a4b"} err="failed to get container status \"a76eea58e2114802ff0e889b1465b5e2c68ad143eb4089836c3ed97389937a4b\": rpc error: code = NotFound desc = could not find container \"a76eea58e2114802ff0e889b1465b5e2c68ad143eb4089836c3ed97389937a4b\": container with ID starting with a76eea58e2114802ff0e889b1465b5e2c68ad143eb4089836c3ed97389937a4b not found: ID does not exist"
Nov 04 02:18:15.925356 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.925157    4865 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-envoy\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.925356 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.925192    4865 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-podinfo\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.925356 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.925213    4865 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c786c4c-0ace-4ac4-aad0-3984cb251c72-istio-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.925356 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.925227    4865 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-istio-envoy\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.925356 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.925245    4865 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3266fdea-562e-4e27-8b95-c1157b563921-istio-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.925356 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.925261    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7xkt\" (UniqueName: \"kubernetes.io/projected/1c786c4c-0ace-4ac4-aad0-3984cb251c72-kube-api-access-r7xkt\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.925356 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.925277    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgj7x\" (UniqueName: \"kubernetes.io/projected/3266fdea-562e-4e27-8b95-c1157b563921-kube-api-access-rgj7x\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.925356 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.925316    4865 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3266fdea-562e-4e27-8b95-c1157b563921-istio-podinfo\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.925356 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:15.925330    4865 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3266fdea-562e-4e27-8b95-c1157b563921-istio-data\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:16.076879 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:16.076782    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"]
Nov 04 02:18:16.080775 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:16.080738    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh"]
Nov 04 02:18:16.193020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:16.192986    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"]
Nov 04 02:18:16.197109 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:16.197079    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j"]
Nov 04 02:18:16.896027 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:16.895941    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c786c4c-0ace-4ac4-aad0-3984cb251c72" path="/var/lib/kubelet/pods/1c786c4c-0ace-4ac4-aad0-3984cb251c72/volumes"
Nov 04 02:18:16.896475 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:16.896461    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3266fdea-562e-4e27-8b95-c1157b563921" path="/var/lib/kubelet/pods/3266fdea-562e-4e27-8b95-c1157b563921/volumes"
Nov 04 02:18:28.404115 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:18:28.404075    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:18:48.139555 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:48.139479    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"]
Nov 04 02:18:48.140088 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:48.139682    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h" podUID="da8fda5b-215f-4c95-9f7f-31146a1fd90d" containerName="maas-api" containerID="cri-o://9f00d25904a619f5195d9cf186e58a0cc5b1d01a791d777563fa266b6695e09a" gracePeriod=30
Nov 04 02:18:48.471384 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:48.471355    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:18:48.569191 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:48.569159    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/da8fda5b-215f-4c95-9f7f-31146a1fd90d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da8fda5b-215f-4c95-9f7f-31146a1fd90d-openshift-service-ca\") pod \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\" (UID: \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\") "
Nov 04 02:18:48.569398 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:48.569224    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/da8fda5b-215f-4c95-9f7f-31146a1fd90d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/da8fda5b-215f-4c95-9f7f-31146a1fd90d-metrics-certs\") pod \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\" (UID: \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\") "
Nov 04 02:18:48.569398 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:48.569298    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/da8fda5b-215f-4c95-9f7f-31146a1fd90d-kube-api-access-fq7ch\" (UniqueName: \"kubernetes.io/projected/da8fda5b-215f-4c95-9f7f-31146a1fd90d-kube-api-access-fq7ch\") pod \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\" (UID: \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\") "
Nov 04 02:18:48.569398 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:48.569352    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/da8fda5b-215f-4c95-9f7f-31146a1fd90d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da8fda5b-215f-4c95-9f7f-31146a1fd90d-maas-api-tls\") pod \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\" (UID: \"da8fda5b-215f-4c95-9f7f-31146a1fd90d\") "
Nov 04 02:18:48.569586 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:48.569548    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da8fda5b-215f-4c95-9f7f-31146a1fd90d-openshift-service-ca" pod "da8fda5b-215f-4c95-9f7f-31146a1fd90d" (UID: "da8fda5b-215f-4c95-9f7f-31146a1fd90d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:18:48.569791 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:48.569765    4865 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da8fda5b-215f-4c95-9f7f-31146a1fd90d-openshift-service-ca\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:48.571484 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:48.571459    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da8fda5b-215f-4c95-9f7f-31146a1fd90d-maas-api-tls" pod "da8fda5b-215f-4c95-9f7f-31146a1fd90d" (UID: "da8fda5b-215f-4c95-9f7f-31146a1fd90d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:18:48.571581 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:48.571565    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da8fda5b-215f-4c95-9f7f-31146a1fd90d-kube-api-access-fq7ch" pod "da8fda5b-215f-4c95-9f7f-31146a1fd90d" (UID: "da8fda5b-215f-4c95-9f7f-31146a1fd90d"). InnerVolumeSpecName "kube-api-access-fq7ch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:18:48.571626 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:48.571584    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da8fda5b-215f-4c95-9f7f-31146a1fd90d-metrics-certs" pod "da8fda5b-215f-4c95-9f7f-31146a1fd90d" (UID: "da8fda5b-215f-4c95-9f7f-31146a1fd90d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:18:48.671028 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:48.670995    4865 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da8fda5b-215f-4c95-9f7f-31146a1fd90d-metrics-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:48.671028 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:48.671023    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fq7ch\" (UniqueName: \"kubernetes.io/projected/da8fda5b-215f-4c95-9f7f-31146a1fd90d-kube-api-access-fq7ch\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:48.671028 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:48.671034    4865 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da8fda5b-215f-4c95-9f7f-31146a1fd90d-maas-api-tls\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:49.189922 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:49.182664    4865 generic.go:356] "Generic (PLEG): container finished" podID="da8fda5b-215f-4c95-9f7f-31146a1fd90d" containerID="9f00d25904a619f5195d9cf186e58a0cc5b1d01a791d777563fa266b6695e09a" exitCode=0
Nov 04 02:18:49.189922 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:49.182897    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h" event={"ID":"da8fda5b-215f-4c95-9f7f-31146a1fd90d","Type":"ContainerDied","Data":"9f00d25904a619f5195d9cf186e58a0cc5b1d01a791d777563fa266b6695e09a"}
Nov 04 02:18:49.189922 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:49.182972    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h" event={"ID":"da8fda5b-215f-4c95-9f7f-31146a1fd90d","Type":"ContainerDied","Data":"7b42b19c66cbdde98a268533ddf49995b1b2f8bbc5ec900580d1519ae5f61e04"}
Nov 04 02:18:49.189922 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:49.183005    4865 scope.go:122] "RemoveContainer" containerID="9f00d25904a619f5195d9cf186e58a0cc5b1d01a791d777563fa266b6695e09a"
Nov 04 02:18:49.189922 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:49.183240    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"
Nov 04 02:18:49.231731 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:49.230863    4865 scope.go:122] "RemoveContainer" containerID="9f00d25904a619f5195d9cf186e58a0cc5b1d01a791d777563fa266b6695e09a"
Nov 04 02:18:49.238441 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:18:49.238410    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f00d25904a619f5195d9cf186e58a0cc5b1d01a791d777563fa266b6695e09a\": container with ID starting with 9f00d25904a619f5195d9cf186e58a0cc5b1d01a791d777563fa266b6695e09a not found: ID does not exist" containerID="9f00d25904a619f5195d9cf186e58a0cc5b1d01a791d777563fa266b6695e09a"
Nov 04 02:18:49.238697 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:49.238659    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f00d25904a619f5195d9cf186e58a0cc5b1d01a791d777563fa266b6695e09a"} err="failed to get container status \"9f00d25904a619f5195d9cf186e58a0cc5b1d01a791d777563fa266b6695e09a\": rpc error: code = NotFound desc = could not find container \"9f00d25904a619f5195d9cf186e58a0cc5b1d01a791d777563fa266b6695e09a\": container with ID starting with 9f00d25904a619f5195d9cf186e58a0cc5b1d01a791d777563fa266b6695e09a not found: ID does not exist"
Nov 04 02:18:49.360362 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:49.360330    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"]
Nov 04 02:18:49.364453 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:49.364428    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a8ef09-698889c764-fpz5h"]
Nov 04 02:18:50.895806 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:50.895766    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="da8fda5b-215f-4c95-9f7f-31146a1fd90d" path="/var/lib/kubelet/pods/da8fda5b-215f-4c95-9f7f-31146a1fd90d/volumes"
Nov 04 02:18:50.899694 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:50.899668    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:18:53.810621 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:53.810584    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"]
Nov 04 02:18:53.811110 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:53.810852    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp" podUID="aeaab295-b237-407a-8b12-24da28c578c0" containerName="istio-proxy" containerID="cri-o://cf011bed5f5fcf89f8b97e180d10aaeb4925b4372a096b4a9cc6a75a45129434" gracePeriod=30
Nov 04 02:18:58.404366 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:18:58.404328    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:18:58.499813 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:58.499780    4865 patch_prober.go:29] interesting pod/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 02:18:58.499967 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:58.499843    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp" podUID="aeaab295-b237-407a-8b12-24da28c578c0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 02:18:59.072972 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.072912    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:18:59.131756 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.131728    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-credential-socket\") pod \"aeaab295-b237-407a-8b12-24da28c578c0\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") "
Nov 04 02:18:59.131904 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.131782    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-istio-data\") pod \"aeaab295-b237-407a-8b12-24da28c578c0\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") "
Nov 04 02:18:59.131904 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.131843    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aeaab295-b237-407a-8b12-24da28c578c0-istio-token\" (UniqueName: \"kubernetes.io/projected/aeaab295-b237-407a-8b12-24da28c578c0-istio-token\") pod \"aeaab295-b237-407a-8b12-24da28c578c0\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") "
Nov 04 02:18:59.131904 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.131897    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-workload-certs\") pod \"aeaab295-b237-407a-8b12-24da28c578c0\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") "
Nov 04 02:18:59.132074 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.131917    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aeaab295-b237-407a-8b12-24da28c578c0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aeaab295-b237-407a-8b12-24da28c578c0-istiod-ca-cert\") pod \"aeaab295-b237-407a-8b12-24da28c578c0\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") "
Nov 04 02:18:59.132074 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.131958    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aeaab295-b237-407a-8b12-24da28c578c0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aeaab295-b237-407a-8b12-24da28c578c0-istio-podinfo\") pod \"aeaab295-b237-407a-8b12-24da28c578c0\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") "
Nov 04 02:18:59.132074 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.131997    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-workload-socket\") pod \"aeaab295-b237-407a-8b12-24da28c578c0\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") "
Nov 04 02:18:59.132074 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.132031    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aeaab295-b237-407a-8b12-24da28c578c0-kube-api-access-btn4b\" (UniqueName: \"kubernetes.io/projected/aeaab295-b237-407a-8b12-24da28c578c0-kube-api-access-btn4b\") pod \"aeaab295-b237-407a-8b12-24da28c578c0\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") "
Nov 04 02:18:59.132074 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.132069    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-istio-envoy\") pod \"aeaab295-b237-407a-8b12-24da28c578c0\" (UID: \"aeaab295-b237-407a-8b12-24da28c578c0\") "
Nov 04 02:18:59.132349 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.132064    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-credential-socket" pod "aeaab295-b237-407a-8b12-24da28c578c0" (UID: "aeaab295-b237-407a-8b12-24da28c578c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:59.132349 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.132102    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-istio-data" pod "aeaab295-b237-407a-8b12-24da28c578c0" (UID: "aeaab295-b237-407a-8b12-24da28c578c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:59.132349 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.132207    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-workload-certs" pod "aeaab295-b237-407a-8b12-24da28c578c0" (UID: "aeaab295-b237-407a-8b12-24da28c578c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:59.132349 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.132309    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-workload-socket" pod "aeaab295-b237-407a-8b12-24da28c578c0" (UID: "aeaab295-b237-407a-8b12-24da28c578c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:59.132543 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.132407    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aeaab295-b237-407a-8b12-24da28c578c0-istiod-ca-cert" pod "aeaab295-b237-407a-8b12-24da28c578c0" (UID: "aeaab295-b237-407a-8b12-24da28c578c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:18:59.132619 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.132596    4865 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-workload-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:59.132678 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.132627    4865 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aeaab295-b237-407a-8b12-24da28c578c0-istiod-ca-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:59.132678 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.132643    4865 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-workload-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:59.132678 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.132658    4865 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-credential-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:59.132678 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.132673    4865 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-istio-data\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:59.134196 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.134172    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aeaab295-b237-407a-8b12-24da28c578c0-istio-podinfo" pod "aeaab295-b237-407a-8b12-24da28c578c0" (UID: "aeaab295-b237-407a-8b12-24da28c578c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:18:59.134325 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.134234    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeaab295-b237-407a-8b12-24da28c578c0-kube-api-access-btn4b" pod "aeaab295-b237-407a-8b12-24da28c578c0" (UID: "aeaab295-b237-407a-8b12-24da28c578c0"). InnerVolumeSpecName "kube-api-access-btn4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:18:59.134325 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.134259    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeaab295-b237-407a-8b12-24da28c578c0-istio-token" pod "aeaab295-b237-407a-8b12-24da28c578c0" (UID: "aeaab295-b237-407a-8b12-24da28c578c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:18:59.134325 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.134270    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-istio-envoy" pod "aeaab295-b237-407a-8b12-24da28c578c0" (UID: "aeaab295-b237-407a-8b12-24da28c578c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:59.233823 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.233794    4865 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aeaab295-b237-407a-8b12-24da28c578c0-istio-podinfo\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:59.233823 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.233822    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btn4b\" (UniqueName: \"kubernetes.io/projected/aeaab295-b237-407a-8b12-24da28c578c0-kube-api-access-btn4b\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:59.233823 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.233834    4865 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aeaab295-b237-407a-8b12-24da28c578c0-istio-envoy\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:59.234077 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.233851    4865 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aeaab295-b237-407a-8b12-24da28c578c0-istio-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:59.240083 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.240057    4865 generic.go:356] "Generic (PLEG): container finished" podID="aeaab295-b237-407a-8b12-24da28c578c0" containerID="cf011bed5f5fcf89f8b97e180d10aaeb4925b4372a096b4a9cc6a75a45129434" exitCode=0
Nov 04 02:18:59.240222 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.240107    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"
Nov 04 02:18:59.240222 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.240145    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp" event={"ID":"aeaab295-b237-407a-8b12-24da28c578c0","Type":"ContainerDied","Data":"cf011bed5f5fcf89f8b97e180d10aaeb4925b4372a096b4a9cc6a75a45129434"}
Nov 04 02:18:59.240222 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.240204    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp" event={"ID":"aeaab295-b237-407a-8b12-24da28c578c0","Type":"ContainerDied","Data":"03d70d56573b81a797d25eaf034306b3d9f9a4f86b7f4525bcf4cd311a344d11"}
Nov 04 02:18:59.240339 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.240226    4865 scope.go:122] "RemoveContainer" containerID="cf011bed5f5fcf89f8b97e180d10aaeb4925b4372a096b4a9cc6a75a45129434"
Nov 04 02:18:59.255268 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.255243    4865 scope.go:122] "RemoveContainer" containerID="435daf6c70478daf0497976300c458eadc3e49e5757d89ae535cb5ba5ff0cfc8"
Nov 04 02:18:59.264460 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.264400    4865 scope.go:122] "RemoveContainer" containerID="cf011bed5f5fcf89f8b97e180d10aaeb4925b4372a096b4a9cc6a75a45129434"
Nov 04 02:18:59.264742 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:18:59.264715    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf011bed5f5fcf89f8b97e180d10aaeb4925b4372a096b4a9cc6a75a45129434\": container with ID starting with cf011bed5f5fcf89f8b97e180d10aaeb4925b4372a096b4a9cc6a75a45129434 not found: ID does not exist" containerID="cf011bed5f5fcf89f8b97e180d10aaeb4925b4372a096b4a9cc6a75a45129434"
Nov 04 02:18:59.264821 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.264753    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf011bed5f5fcf89f8b97e180d10aaeb4925b4372a096b4a9cc6a75a45129434"} err="failed to get container status \"cf011bed5f5fcf89f8b97e180d10aaeb4925b4372a096b4a9cc6a75a45129434\": rpc error: code = NotFound desc = could not find container \"cf011bed5f5fcf89f8b97e180d10aaeb4925b4372a096b4a9cc6a75a45129434\": container with ID starting with cf011bed5f5fcf89f8b97e180d10aaeb4925b4372a096b4a9cc6a75a45129434 not found: ID does not exist"
Nov 04 02:18:59.264821 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.264779    4865 scope.go:122] "RemoveContainer" containerID="435daf6c70478daf0497976300c458eadc3e49e5757d89ae535cb5ba5ff0cfc8"
Nov 04 02:18:59.265110 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:18:59.265090    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"435daf6c70478daf0497976300c458eadc3e49e5757d89ae535cb5ba5ff0cfc8\": container with ID starting with 435daf6c70478daf0497976300c458eadc3e49e5757d89ae535cb5ba5ff0cfc8 not found: ID does not exist" containerID="435daf6c70478daf0497976300c458eadc3e49e5757d89ae535cb5ba5ff0cfc8"
Nov 04 02:18:59.265196 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.265121    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"435daf6c70478daf0497976300c458eadc3e49e5757d89ae535cb5ba5ff0cfc8"} err="failed to get container status \"435daf6c70478daf0497976300c458eadc3e49e5757d89ae535cb5ba5ff0cfc8\": rpc error: code = NotFound desc = could not find container \"435daf6c70478daf0497976300c458eadc3e49e5757d89ae535cb5ba5ff0cfc8\": container with ID starting with 435daf6c70478daf0497976300c458eadc3e49e5757d89ae535cb5ba5ff0cfc8 not found: ID does not exist"
Nov 04 02:18:59.421411 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.421335    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"]
Nov 04 02:18:59.423791 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:18:59.423767    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp"]
Nov 04 02:19:00.397401 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.397362    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"]
Nov 04 02:19:00.397977 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.397956    4865 cpu_manager.go:414] "containerMap: removing container" podUID="1c786c4c-0ace-4ac4-aad0-3984cb251c72" containerName="istio-proxy"
Nov 04 02:19:00.398081 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.397977    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c786c4c-0ace-4ac4-aad0-3984cb251c72" containerName="istio-proxy"
Nov 04 02:19:00.398081 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.397989    4865 cpu_manager.go:414] "containerMap: removing container" podUID="da8fda5b-215f-4c95-9f7f-31146a1fd90d" containerName="maas-api"
Nov 04 02:19:00.398081 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.397998    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="da8fda5b-215f-4c95-9f7f-31146a1fd90d" containerName="maas-api"
Nov 04 02:19:00.398081 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.398010    4865 cpu_manager.go:414] "containerMap: removing container" podUID="3266fdea-562e-4e27-8b95-c1157b563921" containerName="istio-proxy"
Nov 04 02:19:00.398081 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.398018    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3266fdea-562e-4e27-8b95-c1157b563921" containerName="istio-proxy"
Nov 04 02:19:00.398081 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.398065    4865 cpu_manager.go:414] "containerMap: removing container" podUID="1c786c4c-0ace-4ac4-aad0-3984cb251c72" containerName="istio-proxy"
Nov 04 02:19:00.398081 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.398073    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c786c4c-0ace-4ac4-aad0-3984cb251c72" containerName="istio-proxy"
Nov 04 02:19:00.398467 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.398088    4865 cpu_manager.go:414] "containerMap: removing container" podUID="aeaab295-b237-407a-8b12-24da28c578c0" containerName="istio-proxy"
Nov 04 02:19:00.398467 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.398096    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aeaab295-b237-407a-8b12-24da28c578c0" containerName="istio-proxy"
Nov 04 02:19:00.398467 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.398109    4865 cpu_manager.go:414] "containerMap: removing container" podUID="aeaab295-b237-407a-8b12-24da28c578c0" containerName="istio-proxy"
Nov 04 02:19:00.398467 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.398116    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aeaab295-b237-407a-8b12-24da28c578c0" containerName="istio-proxy"
Nov 04 02:19:00.398467 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.398242    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c786c4c-0ace-4ac4-aad0-3984cb251c72" containerName="istio-proxy"
Nov 04 02:19:00.398467 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.398253    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="aeaab295-b237-407a-8b12-24da28c578c0" containerName="istio-proxy"
Nov 04 02:19:00.398467 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.398266    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="aeaab295-b237-407a-8b12-24da28c578c0" containerName="istio-proxy"
Nov 04 02:19:00.398467 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.398296    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="da8fda5b-215f-4c95-9f7f-31146a1fd90d" containerName="maas-api"
Nov 04 02:19:00.398467 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.398304    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="3266fdea-562e-4e27-8b95-c1157b563921" containerName="istio-proxy"
Nov 04 02:19:00.403118 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.403098    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:19:00.448831 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.448799    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bd330e6-5c2c-4095-b862-4e13cec905b4-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh\" (UID: \"0bd330e6-5c2c-4095-b862-4e13cec905b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:19:00.449162 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.448836    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bd330e6-5c2c-4095-b862-4e13cec905b4-metrics-certs\") pod \"maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh\" (UID: \"0bd330e6-5c2c-4095-b862-4e13cec905b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:19:00.449162 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.448987    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bd330e6-5c2c-4095-b862-4e13cec905b4-maas-api-tls\") pod \"maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh\" (UID: \"0bd330e6-5c2c-4095-b862-4e13cec905b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:19:00.449162 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.449049    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6spn\" (UniqueName: \"kubernetes.io/projected/0bd330e6-5c2c-4095-b862-4e13cec905b4-kube-api-access-b6spn\") pod \"maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh\" (UID: \"0bd330e6-5c2c-4095-b862-4e13cec905b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:19:00.457321 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.457277    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-3d082d-serving-cert\""
Nov 04 02:19:00.461762 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.461731    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"]
Nov 04 02:19:00.550242 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.550205    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bd330e6-5c2c-4095-b862-4e13cec905b4-maas-api-tls\") pod \"maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh\" (UID: \"0bd330e6-5c2c-4095-b862-4e13cec905b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:19:00.550433 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.550317    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6spn\" (UniqueName: \"kubernetes.io/projected/0bd330e6-5c2c-4095-b862-4e13cec905b4-kube-api-access-b6spn\") pod \"maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh\" (UID: \"0bd330e6-5c2c-4095-b862-4e13cec905b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:19:00.550433 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:19:00.550398    4865 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-serving-cert: secret "maas-api-e2e-extmod-b-3d082d-serving-cert" not found
Nov 04 02:19:00.550554 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.550440    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bd330e6-5c2c-4095-b862-4e13cec905b4-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh\" (UID: \"0bd330e6-5c2c-4095-b862-4e13cec905b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:19:00.550554 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:19:00.550475    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bd330e6-5c2c-4095-b862-4e13cec905b4-maas-api-tls podName:0bd330e6-5c2c-4095-b862-4e13cec905b4 nodeName:}" failed. No retries permitted until 2026-11-04 02:19:01.050454552 +0000 UTC m=+1404.277241211 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0bd330e6-5c2c-4095-b862-4e13cec905b4-maas-api-tls") pod "maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh" (UID: "0bd330e6-5c2c-4095-b862-4e13cec905b4") : secret "maas-api-e2e-extmod-b-3d082d-serving-cert" not found
Nov 04 02:19:00.550554 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.550513    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bd330e6-5c2c-4095-b862-4e13cec905b4-metrics-certs\") pod \"maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh\" (UID: \"0bd330e6-5c2c-4095-b862-4e13cec905b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:19:00.551094 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.551071    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bd330e6-5c2c-4095-b862-4e13cec905b4-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh\" (UID: \"0bd330e6-5c2c-4095-b862-4e13cec905b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:19:00.553460 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.553436    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bd330e6-5c2c-4095-b862-4e13cec905b4-metrics-certs\") pod \"maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh\" (UID: \"0bd330e6-5c2c-4095-b862-4e13cec905b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:19:00.612672 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.612647    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6spn\" (UniqueName: \"kubernetes.io/projected/0bd330e6-5c2c-4095-b862-4e13cec905b4-kube-api-access-b6spn\") pod \"maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh\" (UID: \"0bd330e6-5c2c-4095-b862-4e13cec905b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:19:00.897156 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:00.897118    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aeaab295-b237-407a-8b12-24da28c578c0" path="/var/lib/kubelet/pods/aeaab295-b237-407a-8b12-24da28c578c0/volumes"
Nov 04 02:19:01.081453 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:01.081420    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bd330e6-5c2c-4095-b862-4e13cec905b4-maas-api-tls\") pod \"maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh\" (UID: \"0bd330e6-5c2c-4095-b862-4e13cec905b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:19:01.083845 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:01.083820    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bd330e6-5c2c-4095-b862-4e13cec905b4-maas-api-tls\") pod \"maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh\" (UID: \"0bd330e6-5c2c-4095-b862-4e13cec905b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:19:01.316385 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:01.316355    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:19:01.693024 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:01.692990    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"]
Nov 04 02:19:01.694632 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:19:01.694605    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bd330e6_5c2c_4095_b862_4e13cec905b4.slice/crio-deccff5f07ba54910101788d355a28d7eb4e57a67295febbfc4d1fc02fd34cb5 WatchSource:0}: Error finding container deccff5f07ba54910101788d355a28d7eb4e57a67295febbfc4d1fc02fd34cb5: Status 404 returned error can't find the container with id deccff5f07ba54910101788d355a28d7eb4e57a67295febbfc4d1fc02fd34cb5
Nov 04 02:19:02.262581 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:02.262524    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh" event={"ID":"0bd330e6-5c2c-4095-b862-4e13cec905b4","Type":"ContainerStarted","Data":"deccff5f07ba54910101788d355a28d7eb4e57a67295febbfc4d1fc02fd34cb5"}
Nov 04 02:19:03.280778 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:03.280735    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh" event={"ID":"0bd330e6-5c2c-4095-b862-4e13cec905b4","Type":"ContainerStarted","Data":"bfb3230f78479833aaa08ce55bbb79ff76aa0a2be7861a2adf8aa8c0aac9922a"}
Nov 04 02:19:03.281277 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:03.280831    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:19:03.408355 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:03.403245    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh" podStartSLOduration=2.452847683 podStartE2EDuration="3.403226408s" totalImagesPullingTime="950.378725ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:19:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:19:03.401670425 +0000 UTC m=+1406.628457106" watchObservedRunningTime="2026-11-04 02:19:03.403226408 +0000 UTC m=+1406.630013078"
Nov 04 02:19:05.208735 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:05.208702    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"]
Nov 04 02:19:05.209163 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:05.208908    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn" podUID="cc3036dd-9c7a-4407-a710-696c360d2e43" containerName="maas-api" containerID="cri-o://a9d38fae3738337f1d526c4dc718d77cdc1d00c4be0d09ebe9409a868093d31a" gracePeriod=30
Nov 04 02:19:05.538698 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:05.538669    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:19:05.663146 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:05.661955    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc3036dd-9c7a-4407-a710-696c360d2e43-kube-api-access-rlcbt\" (UniqueName: \"kubernetes.io/projected/cc3036dd-9c7a-4407-a710-696c360d2e43-kube-api-access-rlcbt\") pod \"cc3036dd-9c7a-4407-a710-696c360d2e43\" (UID: \"cc3036dd-9c7a-4407-a710-696c360d2e43\") "
Nov 04 02:19:05.663146 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:05.662058    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cc3036dd-9c7a-4407-a710-696c360d2e43-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc3036dd-9c7a-4407-a710-696c360d2e43-maas-api-tls\") pod \"cc3036dd-9c7a-4407-a710-696c360d2e43\" (UID: \"cc3036dd-9c7a-4407-a710-696c360d2e43\") "
Nov 04 02:19:05.663146 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:05.662090    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cc3036dd-9c7a-4407-a710-696c360d2e43-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc3036dd-9c7a-4407-a710-696c360d2e43-metrics-certs\") pod \"cc3036dd-9c7a-4407-a710-696c360d2e43\" (UID: \"cc3036dd-9c7a-4407-a710-696c360d2e43\") "
Nov 04 02:19:05.663146 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:05.662117    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc3036dd-9c7a-4407-a710-696c360d2e43-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc3036dd-9c7a-4407-a710-696c360d2e43-openshift-service-ca\") pod \"cc3036dd-9c7a-4407-a710-696c360d2e43\" (UID: \"cc3036dd-9c7a-4407-a710-696c360d2e43\") "
Nov 04 02:19:05.667299 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:05.667228    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc3036dd-9c7a-4407-a710-696c360d2e43-openshift-service-ca" pod "cc3036dd-9c7a-4407-a710-696c360d2e43" (UID: "cc3036dd-9c7a-4407-a710-696c360d2e43"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:19:05.668109 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:05.668053    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc3036dd-9c7a-4407-a710-696c360d2e43-metrics-certs" pod "cc3036dd-9c7a-4407-a710-696c360d2e43" (UID: "cc3036dd-9c7a-4407-a710-696c360d2e43"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:19:05.670705 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:05.668857    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc3036dd-9c7a-4407-a710-696c360d2e43-kube-api-access-rlcbt" pod "cc3036dd-9c7a-4407-a710-696c360d2e43" (UID: "cc3036dd-9c7a-4407-a710-696c360d2e43"). InnerVolumeSpecName "kube-api-access-rlcbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:19:05.671917 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:05.671860    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc3036dd-9c7a-4407-a710-696c360d2e43-maas-api-tls" pod "cc3036dd-9c7a-4407-a710-696c360d2e43" (UID: "cc3036dd-9c7a-4407-a710-696c360d2e43"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:19:05.763967 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:05.763889    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlcbt\" (UniqueName: \"kubernetes.io/projected/cc3036dd-9c7a-4407-a710-696c360d2e43-kube-api-access-rlcbt\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:05.763967 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:05.763930    4865 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc3036dd-9c7a-4407-a710-696c360d2e43-maas-api-tls\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:05.763967 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:05.763946    4865 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc3036dd-9c7a-4407-a710-696c360d2e43-metrics-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:05.763967 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:05.763959    4865 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc3036dd-9c7a-4407-a710-696c360d2e43-openshift-service-ca\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:06.302275 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:06.302239    4865 generic.go:356] "Generic (PLEG): container finished" podID="cc3036dd-9c7a-4407-a710-696c360d2e43" containerID="a9d38fae3738337f1d526c4dc718d77cdc1d00c4be0d09ebe9409a868093d31a" exitCode=0
Nov 04 02:19:06.302745 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:06.302313    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"
Nov 04 02:19:06.302745 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:06.302344    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn" event={"ID":"cc3036dd-9c7a-4407-a710-696c360d2e43","Type":"ContainerDied","Data":"a9d38fae3738337f1d526c4dc718d77cdc1d00c4be0d09ebe9409a868093d31a"}
Nov 04 02:19:06.302745 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:06.302390    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn" event={"ID":"cc3036dd-9c7a-4407-a710-696c360d2e43","Type":"ContainerDied","Data":"ed83d6694cfa4f4d2c5c9b3a349cddb621096f4717769bd3ccedbe7a4f482360"}
Nov 04 02:19:06.302745 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:06.302412    4865 scope.go:122] "RemoveContainer" containerID="a9d38fae3738337f1d526c4dc718d77cdc1d00c4be0d09ebe9409a868093d31a"
Nov 04 02:19:06.312578 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:06.312560    4865 scope.go:122] "RemoveContainer" containerID="a9d38fae3738337f1d526c4dc718d77cdc1d00c4be0d09ebe9409a868093d31a"
Nov 04 02:19:06.312852 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:19:06.312831    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9d38fae3738337f1d526c4dc718d77cdc1d00c4be0d09ebe9409a868093d31a\": container with ID starting with a9d38fae3738337f1d526c4dc718d77cdc1d00c4be0d09ebe9409a868093d31a not found: ID does not exist" containerID="a9d38fae3738337f1d526c4dc718d77cdc1d00c4be0d09ebe9409a868093d31a"
Nov 04 02:19:06.312918 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:06.312864    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9d38fae3738337f1d526c4dc718d77cdc1d00c4be0d09ebe9409a868093d31a"} err="failed to get container status \"a9d38fae3738337f1d526c4dc718d77cdc1d00c4be0d09ebe9409a868093d31a\": rpc error: code = NotFound desc = could not find container \"a9d38fae3738337f1d526c4dc718d77cdc1d00c4be0d09ebe9409a868093d31a\": container with ID starting with a9d38fae3738337f1d526c4dc718d77cdc1d00c4be0d09ebe9409a868093d31a not found: ID does not exist"
Nov 04 02:19:06.475385 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:06.475351    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"]
Nov 04 02:19:06.526167 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:06.526136    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-055a14-fd4d7b54b-lklhn"]
Nov 04 02:19:06.896433 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:06.896402    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc3036dd-9c7a-4407-a710-696c360d2e43" path="/var/lib/kubelet/pods/cc3036dd-9c7a-4407-a710-696c360d2e43/volumes"
Nov 04 02:19:08.922919 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:08.922890    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"]
Nov 04 02:19:08.923382 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:08.923093    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l" podUID="e211ff43-fa6b-4912-9c5b-bc5cec322b03" containerName="istio-proxy" containerID="cri-o://fefcf305f9fc9103f75e5c09f87e53e8cb7ab398ac866b53e550d574755101a5" gracePeriod=30
Nov 04 02:19:09.292446 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:09.292418    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:19:14.201293 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.201251    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:19:14.284080 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.284046    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-workload-socket\") pod \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") "
Nov 04 02:19:14.284249 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.284144    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-envoy\") pod \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") "
Nov 04 02:19:14.284249 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.284231    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e211ff43-fa6b-4912-9c5b-bc5cec322b03-kube-api-access-49zl2\" (UniqueName: \"kubernetes.io/projected/e211ff43-fa6b-4912-9c5b-bc5cec322b03-kube-api-access-49zl2\") pod \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") "
Nov 04 02:19:14.284485 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.284294    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istiod-ca-cert\") pod \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") "
Nov 04 02:19:14.284485 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.284334    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-workload-certs\") pod \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") "
Nov 04 02:19:14.284485 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.284369    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-workload-socket" pod "e211ff43-fa6b-4912-9c5b-bc5cec322b03" (UID: "e211ff43-fa6b-4912-9c5b-bc5cec322b03"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:19:14.284485 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.284379    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-podinfo\") pod \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") "
Nov 04 02:19:14.284485 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.284458    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-token\" (UniqueName: \"kubernetes.io/projected/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-token\") pod \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") "
Nov 04 02:19:14.284757 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.284531    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-data\") pod \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") "
Nov 04 02:19:14.284757 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.284590    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-credential-socket\") pod \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\" (UID: \"e211ff43-fa6b-4912-9c5b-bc5cec322b03\") "
Nov 04 02:19:14.284757 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.284618    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-workload-certs" pod "e211ff43-fa6b-4912-9c5b-bc5cec322b03" (UID: "e211ff43-fa6b-4912-9c5b-bc5cec322b03"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:19:14.284757 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.284667    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istiod-ca-cert" pod "e211ff43-fa6b-4912-9c5b-bc5cec322b03" (UID: "e211ff43-fa6b-4912-9c5b-bc5cec322b03"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:19:14.285009 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.284855    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-data" pod "e211ff43-fa6b-4912-9c5b-bc5cec322b03" (UID: "e211ff43-fa6b-4912-9c5b-bc5cec322b03"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:19:14.285073 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.285043    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-credential-socket" pod "e211ff43-fa6b-4912-9c5b-bc5cec322b03" (UID: "e211ff43-fa6b-4912-9c5b-bc5cec322b03"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:19:14.285150 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.285131    4865 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-workload-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:14.285207 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.285154    4865 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-data\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:14.285207 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.285170    4865 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-credential-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:14.285207 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.285184    4865 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-workload-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:14.285207 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.285194    4865 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istiod-ca-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:14.286641 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.286618    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-podinfo" pod "e211ff43-fa6b-4912-9c5b-bc5cec322b03" (UID: "e211ff43-fa6b-4912-9c5b-bc5cec322b03"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:19:14.286756 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.286656    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e211ff43-fa6b-4912-9c5b-bc5cec322b03-kube-api-access-49zl2" pod "e211ff43-fa6b-4912-9c5b-bc5cec322b03" (UID: "e211ff43-fa6b-4912-9c5b-bc5cec322b03"). InnerVolumeSpecName "kube-api-access-49zl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:19:14.286809 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.286789    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-envoy" pod "e211ff43-fa6b-4912-9c5b-bc5cec322b03" (UID: "e211ff43-fa6b-4912-9c5b-bc5cec322b03"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:19:14.286931 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.286904    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-token" pod "e211ff43-fa6b-4912-9c5b-bc5cec322b03" (UID: "e211ff43-fa6b-4912-9c5b-bc5cec322b03"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:19:14.373593 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.373556    4865 generic.go:356] "Generic (PLEG): container finished" podID="e211ff43-fa6b-4912-9c5b-bc5cec322b03" containerID="fefcf305f9fc9103f75e5c09f87e53e8cb7ab398ac866b53e550d574755101a5" exitCode=0
Nov 04 02:19:14.373767 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.373604    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"
Nov 04 02:19:14.373767 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.373638    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l" event={"ID":"e211ff43-fa6b-4912-9c5b-bc5cec322b03","Type":"ContainerDied","Data":"fefcf305f9fc9103f75e5c09f87e53e8cb7ab398ac866b53e550d574755101a5"}
Nov 04 02:19:14.373767 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.373696    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l" event={"ID":"e211ff43-fa6b-4912-9c5b-bc5cec322b03","Type":"ContainerDied","Data":"945aaad5c431dcf9f0410e74a0f8c6ddde2e0f944ee59ac65c10e6085ace232e"}
Nov 04 02:19:14.373767 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.373726    4865 scope.go:122] "RemoveContainer" containerID="fefcf305f9fc9103f75e5c09f87e53e8cb7ab398ac866b53e550d574755101a5"
Nov 04 02:19:14.385402 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.385375    4865 scope.go:122] "RemoveContainer" containerID="658d4407678b49127be650b8863da502047b56d33b7729eb7df4914af010ddfe"
Nov 04 02:19:14.386063 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.386039    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49zl2\" (UniqueName: \"kubernetes.io/projected/e211ff43-fa6b-4912-9c5b-bc5cec322b03-kube-api-access-49zl2\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:14.386161 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.386066    4865 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-podinfo\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:14.386161 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.386075    4865 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:14.386161 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.386084    4865 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e211ff43-fa6b-4912-9c5b-bc5cec322b03-istio-envoy\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:14.395740 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.395718    4865 scope.go:122] "RemoveContainer" containerID="fefcf305f9fc9103f75e5c09f87e53e8cb7ab398ac866b53e550d574755101a5"
Nov 04 02:19:14.396084 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:19:14.396064    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fefcf305f9fc9103f75e5c09f87e53e8cb7ab398ac866b53e550d574755101a5\": container with ID starting with fefcf305f9fc9103f75e5c09f87e53e8cb7ab398ac866b53e550d574755101a5 not found: ID does not exist" containerID="fefcf305f9fc9103f75e5c09f87e53e8cb7ab398ac866b53e550d574755101a5"
Nov 04 02:19:14.396141 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.396092    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fefcf305f9fc9103f75e5c09f87e53e8cb7ab398ac866b53e550d574755101a5"} err="failed to get container status \"fefcf305f9fc9103f75e5c09f87e53e8cb7ab398ac866b53e550d574755101a5\": rpc error: code = NotFound desc = could not find container \"fefcf305f9fc9103f75e5c09f87e53e8cb7ab398ac866b53e550d574755101a5\": container with ID starting with fefcf305f9fc9103f75e5c09f87e53e8cb7ab398ac866b53e550d574755101a5 not found: ID does not exist"
Nov 04 02:19:14.396141 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.396109    4865 scope.go:122] "RemoveContainer" containerID="658d4407678b49127be650b8863da502047b56d33b7729eb7df4914af010ddfe"
Nov 04 02:19:14.396410 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:19:14.396392    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"658d4407678b49127be650b8863da502047b56d33b7729eb7df4914af010ddfe\": container with ID starting with 658d4407678b49127be650b8863da502047b56d33b7729eb7df4914af010ddfe not found: ID does not exist" containerID="658d4407678b49127be650b8863da502047b56d33b7729eb7df4914af010ddfe"
Nov 04 02:19:14.396453 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.396416    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"658d4407678b49127be650b8863da502047b56d33b7729eb7df4914af010ddfe"} err="failed to get container status \"658d4407678b49127be650b8863da502047b56d33b7729eb7df4914af010ddfe\": rpc error: code = NotFound desc = could not find container \"658d4407678b49127be650b8863da502047b56d33b7729eb7df4914af010ddfe\": container with ID starting with 658d4407678b49127be650b8863da502047b56d33b7729eb7df4914af010ddfe not found: ID does not exist"
Nov 04 02:19:14.547888 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.547802    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"]
Nov 04 02:19:14.550172 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.550148    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l"]
Nov 04 02:19:14.896671 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:14.896595    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e211ff43-fa6b-4912-9c5b-bc5cec322b03" path="/var/lib/kubelet/pods/e211ff43-fa6b-4912-9c5b-bc5cec322b03/volumes"
Nov 04 02:19:16.995443 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:16.995407    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"]
Nov 04 02:19:16.995919 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:16.995906    4865 cpu_manager.go:414] "containerMap: removing container" podUID="e211ff43-fa6b-4912-9c5b-bc5cec322b03" containerName="istio-proxy"
Nov 04 02:19:16.995958 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:16.995920    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e211ff43-fa6b-4912-9c5b-bc5cec322b03" containerName="istio-proxy"
Nov 04 02:19:16.995958 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:16.995934    4865 cpu_manager.go:414] "containerMap: removing container" podUID="cc3036dd-9c7a-4407-a710-696c360d2e43" containerName="maas-api"
Nov 04 02:19:16.995958 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:16.995939    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc3036dd-9c7a-4407-a710-696c360d2e43" containerName="maas-api"
Nov 04 02:19:16.996053 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:16.995958    4865 cpu_manager.go:414] "containerMap: removing container" podUID="e211ff43-fa6b-4912-9c5b-bc5cec322b03" containerName="istio-proxy"
Nov 04 02:19:16.996053 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:16.995964    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e211ff43-fa6b-4912-9c5b-bc5cec322b03" containerName="istio-proxy"
Nov 04 02:19:16.996053 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:16.996042    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c786c4c-0ace-4ac4-aad0-3984cb251c72" containerName="istio-proxy"
Nov 04 02:19:16.996053 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:16.996050    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="e211ff43-fa6b-4912-9c5b-bc5cec322b03" containerName="istio-proxy"
Nov 04 02:19:16.996171 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:16.996057    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc3036dd-9c7a-4407-a710-696c360d2e43" containerName="maas-api"
Nov 04 02:19:17.000706 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.000688    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:19:17.057658 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.057627    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"]
Nov 04 02:19:17.105090 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.105049    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-3f76d0\"/\"agc-cred-82cff89acb3f\""
Nov 04 02:19:17.105260 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.105115    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-3f76d0\"/\"payload-processing-external-model-e2e--0ea04e0a-dockercfg-cksnb\""
Nov 04 02:19:17.105260 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.105191    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-3f76d0\"/\"routing-overlay\""
Nov 04 02:19:17.105260 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.105208    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-3f76d0\"/\"kube-root-ca.crt\""
Nov 04 02:19:17.105260 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.105207    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-3f76d0\"/\"openshift-service-ca.crt\""
Nov 04 02:19:17.105260 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.105253    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-3f76d0\"/\"payload-processing-external-model-plugins-e2e-extmod-b-3f76d0\""
Nov 04 02:19:17.131619 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.131583    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/959ef67d-bb84-4062-aff2-8f69275265b8-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6\" (UID: \"959ef67d-bb84-4062-aff2-8f69275265b8\") " pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:19:17.131619 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.131624    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/959ef67d-bb84-4062-aff2-8f69275265b8-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6\" (UID: \"959ef67d-bb84-4062-aff2-8f69275265b8\") " pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:19:17.131811 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.131651    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/959ef67d-bb84-4062-aff2-8f69275265b8-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6\" (UID: \"959ef67d-bb84-4062-aff2-8f69275265b8\") " pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:19:17.131811 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.131744    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb5s2\" (UniqueName: \"kubernetes.io/projected/959ef67d-bb84-4062-aff2-8f69275265b8-kube-api-access-hb5s2\") pod \"payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6\" (UID: \"959ef67d-bb84-4062-aff2-8f69275265b8\") " pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:19:17.233277 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.233232    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/959ef67d-bb84-4062-aff2-8f69275265b8-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6\" (UID: \"959ef67d-bb84-4062-aff2-8f69275265b8\") " pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:19:17.233487 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.233313    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/959ef67d-bb84-4062-aff2-8f69275265b8-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6\" (UID: \"959ef67d-bb84-4062-aff2-8f69275265b8\") " pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:19:17.233487 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.233347    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/959ef67d-bb84-4062-aff2-8f69275265b8-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6\" (UID: \"959ef67d-bb84-4062-aff2-8f69275265b8\") " pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:19:17.233487 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.233386    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb5s2\" (UniqueName: \"kubernetes.io/projected/959ef67d-bb84-4062-aff2-8f69275265b8-kube-api-access-hb5s2\") pod \"payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6\" (UID: \"959ef67d-bb84-4062-aff2-8f69275265b8\") " pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:19:17.234082 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.234055    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/959ef67d-bb84-4062-aff2-8f69275265b8-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6\" (UID: \"959ef67d-bb84-4062-aff2-8f69275265b8\") " pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:19:17.234239 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.234218    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/959ef67d-bb84-4062-aff2-8f69275265b8-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6\" (UID: \"959ef67d-bb84-4062-aff2-8f69275265b8\") " pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:19:17.235673 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.235653    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/959ef67d-bb84-4062-aff2-8f69275265b8-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6\" (UID: \"959ef67d-bb84-4062-aff2-8f69275265b8\") " pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:19:17.295664 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.295594    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb5s2\" (UniqueName: \"kubernetes.io/projected/959ef67d-bb84-4062-aff2-8f69275265b8-kube-api-access-hb5s2\") pod \"payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6\" (UID: \"959ef67d-bb84-4062-aff2-8f69275265b8\") " pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:19:17.313505 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.313475    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:19:17.689413 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:17.689385    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"]
Nov 04 02:19:17.690836 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:19:17.690813    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod959ef67d_bb84_4062_aff2_8f69275265b8.slice/crio-566b8e28c6a3cff18e042d8964a1289fc695ae05c4e971c475102aced8bb45f2 WatchSource:0}: Error finding container 566b8e28c6a3cff18e042d8964a1289fc695ae05c4e971c475102aced8bb45f2: Status 404 returned error can't find the container with id 566b8e28c6a3cff18e042d8964a1289fc695ae05c4e971c475102aced8bb45f2
Nov 04 02:19:18.402778 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:18.402736    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6" event={"ID":"959ef67d-bb84-4062-aff2-8f69275265b8","Type":"ContainerStarted","Data":"fbbaeefd0d163583b296f3949351a7796364b42578c8759f490c92a2a1150cab"}
Nov 04 02:19:18.403211 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:18.402795    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6" event={"ID":"959ef67d-bb84-4062-aff2-8f69275265b8","Type":"ContainerStarted","Data":"566b8e28c6a3cff18e042d8964a1289fc695ae05c4e971c475102aced8bb45f2"}
Nov 04 02:19:18.403211 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:18.402947    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:19:18.521650 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:18.521587    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6" podStartSLOduration=2.521571748 podStartE2EDuration="2.521571748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:19:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:19:18.52095601 +0000 UTC m=+1421.747742690" watchObservedRunningTime="2026-11-04 02:19:18.521571748 +0000 UTC m=+1421.748358416"
Nov 04 02:19:20.706756 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:20.706723    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:19:28.404809 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:19:28.404768    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:19:29.412714 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:29.412677    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:19:31.618616 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:31.618582    4865 generic.go:356] "Generic (PLEG): container finished" podID="707f17bc-7327-4cd7-9796-497331e55d9c" containerID="d1e597d977231f66f057dc7f061c3ef64c330640ccbab84515f07b9b7b868170" exitCode=0
Nov 04 02:19:31.619044 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:31.618657    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd" event={"ID":"707f17bc-7327-4cd7-9796-497331e55d9c","Type":"ContainerDied","Data":"d1e597d977231f66f057dc7f061c3ef64c330640ccbab84515f07b9b7b868170"}
Nov 04 02:19:31.619109 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:31.619094    4865 scope.go:122] "RemoveContainer" containerID="d1e597d977231f66f057dc7f061c3ef64c330640ccbab84515f07b9b7b868170"
Nov 04 02:19:31.620617 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:31.620592    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:19:31.620690 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:31.620659    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:19:31.620726 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:31.620693    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:19:32.630575 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:32.630534    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd" event={"ID":"707f17bc-7327-4cd7-9796-497331e55d9c","Type":"ContainerStarted","Data":"4041240c8fd95cd9b23a4ebb2fdc399e94e127ff3e4209654e0d361addf032e7"}
Nov 04 02:19:32.663179 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:32.663144    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:19:32.663438 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:32.663413    4865 patch_prober.go:29] interesting pod/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 02:19:32.663495 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:32.663477    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd" podUID="707f17bc-7327-4cd7-9796-497331e55d9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 02:19:33.663930 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:33.663896    4865 patch_prober.go:29] interesting pod/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 02:19:33.664362 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:33.663947    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd" podUID="707f17bc-7327-4cd7-9796-497331e55d9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 02:19:34.663798 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:34.663765    4865 patch_prober.go:29] interesting pod/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 02:19:34.663982 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:34.663816    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd" podUID="707f17bc-7327-4cd7-9796-497331e55d9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 02:19:35.663798 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:35.663760    4865 patch_prober.go:29] interesting pod/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 02:19:35.663993 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:35.663826    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd" podUID="707f17bc-7327-4cd7-9796-497331e55d9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 02:19:36.663842 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:36.663802    4865 patch_prober.go:29] interesting pod/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 02:19:36.664029 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:36.663861    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd" podUID="707f17bc-7327-4cd7-9796-497331e55d9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 02:19:37.664351 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:37.664317    4865 patch_prober.go:29] interesting pod/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 02:19:37.664853 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:37.664823    4865 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd" podUID="707f17bc-7327-4cd7-9796-497331e55d9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 02:19:38.667773 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:38.667739    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:19:38.668211 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:38.668200    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:19:38.668806 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:38.668785    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:19:41.883666 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:41.883630    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"]
Nov 04 02:19:41.884324 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:41.883871    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct" podUID="ee0fef40-6d80-40f1-9c16-5d92fc9759b0" containerName="maas-api" containerID="cri-o://86957a21ad5dcde15e07097e6fa8ea36ddab74f0bc5bdcab34ff8e0b572d0e05" gracePeriod=30
Nov 04 02:19:42.205809 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.205782    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:19:42.293973 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.291541    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-kube-api-access-vrd2v\" (UniqueName: \"kubernetes.io/projected/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-kube-api-access-vrd2v\") pod \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\" (UID: \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\") "
Nov 04 02:19:42.293973 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.291596    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-openshift-service-ca\") pod \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\" (UID: \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\") "
Nov 04 02:19:42.293973 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.291664    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-metrics-certs\") pod \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\" (UID: \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\") "
Nov 04 02:19:42.293973 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.291867    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-maas-api-tls\") pod \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\" (UID: \"ee0fef40-6d80-40f1-9c16-5d92fc9759b0\") "
Nov 04 02:19:42.297327 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.294608    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-openshift-service-ca" pod "ee0fef40-6d80-40f1-9c16-5d92fc9759b0" (UID: "ee0fef40-6d80-40f1-9c16-5d92fc9759b0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:19:42.309334 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.307522    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-metrics-certs" pod "ee0fef40-6d80-40f1-9c16-5d92fc9759b0" (UID: "ee0fef40-6d80-40f1-9c16-5d92fc9759b0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:19:42.309334 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.307642    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-kube-api-access-vrd2v" pod "ee0fef40-6d80-40f1-9c16-5d92fc9759b0" (UID: "ee0fef40-6d80-40f1-9c16-5d92fc9759b0"). InnerVolumeSpecName "kube-api-access-vrd2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:19:42.309576 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.309523    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-maas-api-tls" pod "ee0fef40-6d80-40f1-9c16-5d92fc9759b0" (UID: "ee0fef40-6d80-40f1-9c16-5d92fc9759b0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:19:42.393881 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.393842    4865 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-metrics-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:42.393881 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.393881    4865 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-maas-api-tls\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:42.394086 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.393895    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrd2v\" (UniqueName: \"kubernetes.io/projected/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-kube-api-access-vrd2v\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:42.394086 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.393909    4865 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee0fef40-6d80-40f1-9c16-5d92fc9759b0-openshift-service-ca\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:42.731900 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.731861    4865 generic.go:356] "Generic (PLEG): container finished" podID="ee0fef40-6d80-40f1-9c16-5d92fc9759b0" containerID="86957a21ad5dcde15e07097e6fa8ea36ddab74f0bc5bdcab34ff8e0b572d0e05" exitCode=0
Nov 04 02:19:42.732078 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.731903    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"
Nov 04 02:19:42.732078 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.731980    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct" event={"ID":"ee0fef40-6d80-40f1-9c16-5d92fc9759b0","Type":"ContainerDied","Data":"86957a21ad5dcde15e07097e6fa8ea36ddab74f0bc5bdcab34ff8e0b572d0e05"}
Nov 04 02:19:42.732078 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.732024    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct" event={"ID":"ee0fef40-6d80-40f1-9c16-5d92fc9759b0","Type":"ContainerDied","Data":"f5c6236a77581951b88e31db2522cc55eb84b3bcab3f832972c5e2cf38f27358"}
Nov 04 02:19:42.732078 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.732047    4865 scope.go:122] "RemoveContainer" containerID="86957a21ad5dcde15e07097e6fa8ea36ddab74f0bc5bdcab34ff8e0b572d0e05"
Nov 04 02:19:42.743559 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.743536    4865 scope.go:122] "RemoveContainer" containerID="86957a21ad5dcde15e07097e6fa8ea36ddab74f0bc5bdcab34ff8e0b572d0e05"
Nov 04 02:19:42.743835 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:19:42.743817    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86957a21ad5dcde15e07097e6fa8ea36ddab74f0bc5bdcab34ff8e0b572d0e05\": container with ID starting with 86957a21ad5dcde15e07097e6fa8ea36ddab74f0bc5bdcab34ff8e0b572d0e05 not found: ID does not exist" containerID="86957a21ad5dcde15e07097e6fa8ea36ddab74f0bc5bdcab34ff8e0b572d0e05"
Nov 04 02:19:42.743882 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.743842    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86957a21ad5dcde15e07097e6fa8ea36ddab74f0bc5bdcab34ff8e0b572d0e05"} err="failed to get container status \"86957a21ad5dcde15e07097e6fa8ea36ddab74f0bc5bdcab34ff8e0b572d0e05\": rpc error: code = NotFound desc = could not find container \"86957a21ad5dcde15e07097e6fa8ea36ddab74f0bc5bdcab34ff8e0b572d0e05\": container with ID starting with 86957a21ad5dcde15e07097e6fa8ea36ddab74f0bc5bdcab34ff8e0b572d0e05 not found: ID does not exist"
Nov 04 02:19:42.753891 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:19:42.753868    4865 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee0fef40_6d80_40f1_9c16_5d92fc9759b0.slice\": RecentStats: unable to find data in memory cache]"
Nov 04 02:19:42.931023 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.930987    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"]
Nov 04 02:19:42.982105 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:42.982011    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3bf869-6c54994f79-59lct"]
Nov 04 02:19:44.897476 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:44.897431    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee0fef40-6d80-40f1-9c16-5d92fc9759b0" path="/var/lib/kubelet/pods/ee0fef40-6d80-40f1-9c16-5d92fc9759b0/volumes"
Nov 04 02:19:55.499240 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:55.499204    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"]
Nov 04 02:19:55.499706 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:19:55.499501    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9" podUID="8c7a83a4-3356-417d-ad04-f258a97655fd" containerName="istio-proxy" containerID="cri-o://ee3630ac199e04f5e431e39987e58e025aec01ad876ebb58362a6e05c807548c" gracePeriod=30
Nov 04 02:19:58.404516 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:19:58.404471    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:20:00.761116 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.761087    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:20:00.858422 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.858327    4865 generic.go:356] "Generic (PLEG): container finished" podID="8c7a83a4-3356-417d-ad04-f258a97655fd" containerID="ee3630ac199e04f5e431e39987e58e025aec01ad876ebb58362a6e05c807548c" exitCode=0
Nov 04 02:20:00.858422 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.858375    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"
Nov 04 02:20:00.858422 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.858404    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9" event={"ID":"8c7a83a4-3356-417d-ad04-f258a97655fd","Type":"ContainerDied","Data":"ee3630ac199e04f5e431e39987e58e025aec01ad876ebb58362a6e05c807548c"}
Nov 04 02:20:00.858662 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.858470    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9" event={"ID":"8c7a83a4-3356-417d-ad04-f258a97655fd","Type":"ContainerDied","Data":"bf949a8c2e4288c4c87a0bb63c35786ff9bfc7751fb46cd96b1428e229112b10"}
Nov 04 02:20:00.858662 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.858494    4865 scope.go:122] "RemoveContainer" containerID="ee3630ac199e04f5e431e39987e58e025aec01ad876ebb58362a6e05c807548c"
Nov 04 02:20:00.862396 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.862374    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-workload-socket\") pod \"8c7a83a4-3356-417d-ad04-f258a97655fd\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") "
Nov 04 02:20:00.862502 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.862447    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-podinfo\") pod \"8c7a83a4-3356-417d-ad04-f258a97655fd\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") "
Nov 04 02:20:00.862592 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.862511    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-envoy\") pod \"8c7a83a4-3356-417d-ad04-f258a97655fd\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") "
Nov 04 02:20:00.862592 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.862537    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-workload-certs\") pod \"8c7a83a4-3356-417d-ad04-f258a97655fd\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") "
Nov 04 02:20:00.862691 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.862613    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-credential-socket\") pod \"8c7a83a4-3356-417d-ad04-f258a97655fd\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") "
Nov 04 02:20:00.862691 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.862653    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-workload-socket" pod "8c7a83a4-3356-417d-ad04-f258a97655fd" (UID: "8c7a83a4-3356-417d-ad04-f258a97655fd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:20:00.862691 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.862665    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c7a83a4-3356-417d-ad04-f258a97655fd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c7a83a4-3356-417d-ad04-f258a97655fd-istiod-ca-cert\") pod \"8c7a83a4-3356-417d-ad04-f258a97655fd\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") "
Nov 04 02:20:00.862831 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.862731    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-token\" (UniqueName: \"kubernetes.io/projected/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-token\") pod \"8c7a83a4-3356-417d-ad04-f258a97655fd\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") "
Nov 04 02:20:00.862831 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.862763    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c7a83a4-3356-417d-ad04-f258a97655fd-kube-api-access-xddq5\" (UniqueName: \"kubernetes.io/projected/8c7a83a4-3356-417d-ad04-f258a97655fd-kube-api-access-xddq5\") pod \"8c7a83a4-3356-417d-ad04-f258a97655fd\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") "
Nov 04 02:20:00.862936 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.862839    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-data\") pod \"8c7a83a4-3356-417d-ad04-f258a97655fd\" (UID: \"8c7a83a4-3356-417d-ad04-f258a97655fd\") "
Nov 04 02:20:00.862936 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.862849    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-workload-certs" pod "8c7a83a4-3356-417d-ad04-f258a97655fd" (UID: "8c7a83a4-3356-417d-ad04-f258a97655fd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:20:00.863121 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.863082    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c7a83a4-3356-417d-ad04-f258a97655fd-istiod-ca-cert" pod "8c7a83a4-3356-417d-ad04-f258a97655fd" (UID: "8c7a83a4-3356-417d-ad04-f258a97655fd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:20:00.863121 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.863115    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-credential-socket" pod "8c7a83a4-3356-417d-ad04-f258a97655fd" (UID: "8c7a83a4-3356-417d-ad04-f258a97655fd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:20:00.863491 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.863384    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-data" pod "8c7a83a4-3356-417d-ad04-f258a97655fd" (UID: "8c7a83a4-3356-417d-ad04-f258a97655fd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:20:00.863491 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.863484    4865 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-workload-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:00.863668 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.863506    4865 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-credential-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:00.863668 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.863520    4865 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c7a83a4-3356-417d-ad04-f258a97655fd-istiod-ca-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:00.863668 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.863533    4865 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-data\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:00.863668 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.863546    4865 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-workload-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:00.864886 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.864865    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-envoy" pod "8c7a83a4-3356-417d-ad04-f258a97655fd" (UID: "8c7a83a4-3356-417d-ad04-f258a97655fd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:20:00.865037 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.865012    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-token" pod "8c7a83a4-3356-417d-ad04-f258a97655fd" (UID: "8c7a83a4-3356-417d-ad04-f258a97655fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:20:00.865083 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.865051    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-podinfo" pod "8c7a83a4-3356-417d-ad04-f258a97655fd" (UID: "8c7a83a4-3356-417d-ad04-f258a97655fd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:20:00.868383 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.868353    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c7a83a4-3356-417d-ad04-f258a97655fd-kube-api-access-xddq5" pod "8c7a83a4-3356-417d-ad04-f258a97655fd" (UID: "8c7a83a4-3356-417d-ad04-f258a97655fd"). InnerVolumeSpecName "kube-api-access-xddq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:20:00.874397 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.872517    4865 scope.go:122] "RemoveContainer" containerID="d77f2addb0847cfd79dcf0d565a2daa03eb8e9ef581a562488ca288589e9bf61"
Nov 04 02:20:00.893035 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.893011    4865 scope.go:122] "RemoveContainer" containerID="ee3630ac199e04f5e431e39987e58e025aec01ad876ebb58362a6e05c807548c"
Nov 04 02:20:00.893395 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:20:00.893373    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee3630ac199e04f5e431e39987e58e025aec01ad876ebb58362a6e05c807548c\": container with ID starting with ee3630ac199e04f5e431e39987e58e025aec01ad876ebb58362a6e05c807548c not found: ID does not exist" containerID="ee3630ac199e04f5e431e39987e58e025aec01ad876ebb58362a6e05c807548c"
Nov 04 02:20:00.893473 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.893405    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee3630ac199e04f5e431e39987e58e025aec01ad876ebb58362a6e05c807548c"} err="failed to get container status \"ee3630ac199e04f5e431e39987e58e025aec01ad876ebb58362a6e05c807548c\": rpc error: code = NotFound desc = could not find container \"ee3630ac199e04f5e431e39987e58e025aec01ad876ebb58362a6e05c807548c\": container with ID starting with ee3630ac199e04f5e431e39987e58e025aec01ad876ebb58362a6e05c807548c not found: ID does not exist"
Nov 04 02:20:00.893473 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.893424    4865 scope.go:122] "RemoveContainer" containerID="d77f2addb0847cfd79dcf0d565a2daa03eb8e9ef581a562488ca288589e9bf61"
Nov 04 02:20:00.893755 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:20:00.893728    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d77f2addb0847cfd79dcf0d565a2daa03eb8e9ef581a562488ca288589e9bf61\": container with ID starting with d77f2addb0847cfd79dcf0d565a2daa03eb8e9ef581a562488ca288589e9bf61 not found: ID does not exist" containerID="d77f2addb0847cfd79dcf0d565a2daa03eb8e9ef581a562488ca288589e9bf61"
Nov 04 02:20:00.893799 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.893768    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d77f2addb0847cfd79dcf0d565a2daa03eb8e9ef581a562488ca288589e9bf61"} err="failed to get container status \"d77f2addb0847cfd79dcf0d565a2daa03eb8e9ef581a562488ca288589e9bf61\": rpc error: code = NotFound desc = could not find container \"d77f2addb0847cfd79dcf0d565a2daa03eb8e9ef581a562488ca288589e9bf61\": container with ID starting with d77f2addb0847cfd79dcf0d565a2daa03eb8e9ef581a562488ca288589e9bf61 not found: ID does not exist"
Nov 04 02:20:00.964517 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.964475    4865 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-podinfo\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:00.964517 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.964519    4865 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-envoy\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:00.964736 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.964536    4865 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c7a83a4-3356-417d-ad04-f258a97655fd-istio-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:00.964736 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:00.964550    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xddq5\" (UniqueName: \"kubernetes.io/projected/8c7a83a4-3356-417d-ad04-f258a97655fd-kube-api-access-xddq5\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:01.276265 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:01.276227    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"]
Nov 04 02:20:01.280587 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:01.280559    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9"]
Nov 04 02:20:02.896553 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:02.896519    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c7a83a4-3356-417d-ad04-f258a97655fd" path="/var/lib/kubelet/pods/8c7a83a4-3356-417d-ad04-f258a97655fd/volumes"
Nov 04 02:20:04.040133 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:04.040101    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:20:07.330472 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:07.330430    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"]
Nov 04 02:20:07.330993 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:07.330774    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2" podUID="746c1e17-96d9-40fd-8cce-8aebfbc17666" containerName="istio-proxy" containerID="cri-o://a3f20a0df7b7913dbb4c9759dab1a8ae34d90512cfca64e2446dfbdac18875d7" gracePeriod=30
Nov 04 02:20:09.086705 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.086664    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"]
Nov 04 02:20:09.087180 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.087166    4865 cpu_manager.go:414] "containerMap: removing container" podUID="8c7a83a4-3356-417d-ad04-f258a97655fd" containerName="istio-proxy"
Nov 04 02:20:09.087247 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.087181    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c7a83a4-3356-417d-ad04-f258a97655fd" containerName="istio-proxy"
Nov 04 02:20:09.087247 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.087202    4865 cpu_manager.go:414] "containerMap: removing container" podUID="8c7a83a4-3356-417d-ad04-f258a97655fd" containerName="istio-proxy"
Nov 04 02:20:09.087247 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.087208    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c7a83a4-3356-417d-ad04-f258a97655fd" containerName="istio-proxy"
Nov 04 02:20:09.087247 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.087233    4865 cpu_manager.go:414] "containerMap: removing container" podUID="ee0fef40-6d80-40f1-9c16-5d92fc9759b0" containerName="maas-api"
Nov 04 02:20:09.087247 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.087239    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee0fef40-6d80-40f1-9c16-5d92fc9759b0" containerName="maas-api"
Nov 04 02:20:09.087439 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.087319    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee0fef40-6d80-40f1-9c16-5d92fc9759b0" containerName="maas-api"
Nov 04 02:20:09.087439 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.087327    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="e211ff43-fa6b-4912-9c5b-bc5cec322b03" containerName="istio-proxy"
Nov 04 02:20:09.087439 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.087336    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c7a83a4-3356-417d-ad04-f258a97655fd" containerName="istio-proxy"
Nov 04 02:20:09.090815 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.090792    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:20:09.145848 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.145816    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-3d082d\"/\"payload-processing-external-model-plugins-e2e-extmod-b-3d082d\""
Nov 04 02:20:09.146012 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.145858    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-3d082d\"/\"routing-overlay\""
Nov 04 02:20:09.226327 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.221662    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-3d082d\"/\"agc-cred-6b0ffb90bac3\""
Nov 04 02:20:09.226327 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.221818    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-3d082d\"/\"kube-root-ca.crt\""
Nov 04 02:20:09.226327 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.221884    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-3d082d\"/\"openshift-service-ca.crt\""
Nov 04 02:20:09.226327 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.222053    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-3d082d\"/\"payload-processing-external-model-e2e--a01fe53f-dockercfg-kljh2\""
Nov 04 02:20:09.228618 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.228556    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/3a034e7b-5c1c-49b8-a850-cac01d4ec932-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb\" (UID: \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\") " pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:20:09.228843 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.228676    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/3a034e7b-5c1c-49b8-a850-cac01d4ec932-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb\" (UID: \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\") " pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:20:09.228843 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.228747    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a034e7b-5c1c-49b8-a850-cac01d4ec932-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb\" (UID: \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\") " pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:20:09.228843 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.228813    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt9xj\" (UniqueName: \"kubernetes.io/projected/3a034e7b-5c1c-49b8-a850-cac01d4ec932-kube-api-access-rt9xj\") pod \"payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb\" (UID: \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\") " pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:20:09.242488 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.242454    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"]
Nov 04 02:20:09.330312 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.330268    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt9xj\" (UniqueName: \"kubernetes.io/projected/3a034e7b-5c1c-49b8-a850-cac01d4ec932-kube-api-access-rt9xj\") pod \"payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb\" (UID: \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\") " pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:20:09.330483 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.330380    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/3a034e7b-5c1c-49b8-a850-cac01d4ec932-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb\" (UID: \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\") " pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:20:09.330483 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.330453    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/3a034e7b-5c1c-49b8-a850-cac01d4ec932-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb\" (UID: \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\") " pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:20:09.330565 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.330506    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a034e7b-5c1c-49b8-a850-cac01d4ec932-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb\" (UID: \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\") " pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:20:09.331147 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.331120    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/3a034e7b-5c1c-49b8-a850-cac01d4ec932-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb\" (UID: \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\") " pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:20:09.331258 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.331214    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a034e7b-5c1c-49b8-a850-cac01d4ec932-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb\" (UID: \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\") " pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:20:09.332831 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.332810    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/3a034e7b-5c1c-49b8-a850-cac01d4ec932-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb\" (UID: \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\") " pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:20:09.393004 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.392931    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt9xj\" (UniqueName: \"kubernetes.io/projected/3a034e7b-5c1c-49b8-a850-cac01d4ec932-kube-api-access-rt9xj\") pod \"payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb\" (UID: \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\") " pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:20:09.404645 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.404611    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:20:09.783443 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.783409    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"]
Nov 04 02:20:09.783960 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:20:09.783933    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a034e7b_5c1c_49b8_a850_cac01d4ec932.slice/crio-b6b398315a5690f4f720c67fc9d12cb6ad7ee25794c071052d365a7cfe971ff2 WatchSource:0}: Error finding container b6b398315a5690f4f720c67fc9d12cb6ad7ee25794c071052d365a7cfe971ff2: Status 404 returned error can't find the container with id b6b398315a5690f4f720c67fc9d12cb6ad7ee25794c071052d365a7cfe971ff2
Nov 04 02:20:09.972567 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.972523    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb" event={"ID":"3a034e7b-5c1c-49b8-a850-cac01d4ec932","Type":"ContainerStarted","Data":"4b32c44132dab2c482463f94c8293bc4029c4901c6efdc42fbe12ebbf768c61e"}
Nov 04 02:20:09.972735 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.972592    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:20:09.972735 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:09.972613    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb" event={"ID":"3a034e7b-5c1c-49b8-a850-cac01d4ec932","Type":"ContainerStarted","Data":"b6b398315a5690f4f720c67fc9d12cb6ad7ee25794c071052d365a7cfe971ff2"}
Nov 04 02:20:10.095973 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:10.095861    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb" podStartSLOduration=1.095841115 podStartE2EDuration="1.095841115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:20:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:20:10.093878269 +0000 UTC m=+1473.320664928" watchObservedRunningTime="2026-11-04 02:20:10.095841115 +0000 UTC m=+1473.322627780"
Nov 04 02:20:12.598920 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.598857    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:20:12.672407 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.672372    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-envoy\") pod \"746c1e17-96d9-40fd-8cce-8aebfbc17666\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") "
Nov 04 02:20:12.672582 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.672414    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-token\" (UniqueName: \"kubernetes.io/projected/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-token\") pod \"746c1e17-96d9-40fd-8cce-8aebfbc17666\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") "
Nov 04 02:20:12.672582 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.672440    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-workload-certs\") pod \"746c1e17-96d9-40fd-8cce-8aebfbc17666\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") "
Nov 04 02:20:12.672582 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.672493    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-workload-socket\") pod \"746c1e17-96d9-40fd-8cce-8aebfbc17666\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") "
Nov 04 02:20:12.672750 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.672579    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-podinfo\") pod \"746c1e17-96d9-40fd-8cce-8aebfbc17666\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") "
Nov 04 02:20:12.672750 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.672619    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-credential-socket\") pod \"746c1e17-96d9-40fd-8cce-8aebfbc17666\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") "
Nov 04 02:20:12.672750 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.672699    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/746c1e17-96d9-40fd-8cce-8aebfbc17666-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/746c1e17-96d9-40fd-8cce-8aebfbc17666-istiod-ca-cert\") pod \"746c1e17-96d9-40fd-8cce-8aebfbc17666\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") "
Nov 04 02:20:12.672750 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.672744    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-data\") pod \"746c1e17-96d9-40fd-8cce-8aebfbc17666\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") "
Nov 04 02:20:12.672940 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.672844    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-workload-socket" pod "746c1e17-96d9-40fd-8cce-8aebfbc17666" (UID: "746c1e17-96d9-40fd-8cce-8aebfbc17666"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:20:12.672940 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.672857    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-workload-certs" pod "746c1e17-96d9-40fd-8cce-8aebfbc17666" (UID: "746c1e17-96d9-40fd-8cce-8aebfbc17666"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:20:12.672940 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.672874    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/746c1e17-96d9-40fd-8cce-8aebfbc17666-kube-api-access-qk2rb\" (UniqueName: \"kubernetes.io/projected/746c1e17-96d9-40fd-8cce-8aebfbc17666-kube-api-access-qk2rb\") pod \"746c1e17-96d9-40fd-8cce-8aebfbc17666\" (UID: \"746c1e17-96d9-40fd-8cce-8aebfbc17666\") "
Nov 04 02:20:12.673102 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.673086    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-data" pod "746c1e17-96d9-40fd-8cce-8aebfbc17666" (UID: "746c1e17-96d9-40fd-8cce-8aebfbc17666"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:20:12.673180 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.672975    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-credential-socket" pod "746c1e17-96d9-40fd-8cce-8aebfbc17666" (UID: "746c1e17-96d9-40fd-8cce-8aebfbc17666"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:20:12.673227 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.673176    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/746c1e17-96d9-40fd-8cce-8aebfbc17666-istiod-ca-cert" pod "746c1e17-96d9-40fd-8cce-8aebfbc17666" (UID: "746c1e17-96d9-40fd-8cce-8aebfbc17666"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:20:12.673536 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.673510    4865 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-workload-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:12.673651 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.673540    4865 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-workload-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:12.673651 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.673555    4865 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-credential-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:12.673651 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.673569    4865 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/746c1e17-96d9-40fd-8cce-8aebfbc17666-istiod-ca-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:12.673651 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.673584    4865 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-data\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:12.674704 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.674677    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-token" pod "746c1e17-96d9-40fd-8cce-8aebfbc17666" (UID: "746c1e17-96d9-40fd-8cce-8aebfbc17666"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:20:12.674785 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.674686    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-envoy" pod "746c1e17-96d9-40fd-8cce-8aebfbc17666" (UID: "746c1e17-96d9-40fd-8cce-8aebfbc17666"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:20:12.674939 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.674915    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-podinfo" pod "746c1e17-96d9-40fd-8cce-8aebfbc17666" (UID: "746c1e17-96d9-40fd-8cce-8aebfbc17666"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:20:12.675245 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.675227    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/746c1e17-96d9-40fd-8cce-8aebfbc17666-kube-api-access-qk2rb" pod "746c1e17-96d9-40fd-8cce-8aebfbc17666" (UID: "746c1e17-96d9-40fd-8cce-8aebfbc17666"). InnerVolumeSpecName "kube-api-access-qk2rb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:20:12.774527 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.774482    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qk2rb\" (UniqueName: \"kubernetes.io/projected/746c1e17-96d9-40fd-8cce-8aebfbc17666-kube-api-access-qk2rb\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:12.774527 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.774516    4865 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-envoy\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:12.774527 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.774529    4865 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:12.774931 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.774544    4865 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/746c1e17-96d9-40fd-8cce-8aebfbc17666-istio-podinfo\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:12.991934 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.991891    4865 generic.go:356] "Generic (PLEG): container finished" podID="746c1e17-96d9-40fd-8cce-8aebfbc17666" containerID="a3f20a0df7b7913dbb4c9759dab1a8ae34d90512cfca64e2446dfbdac18875d7" exitCode=0
Nov 04 02:20:12.991934 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.991928    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"
Nov 04 02:20:12.992191 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.991968    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2" event={"ID":"746c1e17-96d9-40fd-8cce-8aebfbc17666","Type":"ContainerDied","Data":"a3f20a0df7b7913dbb4c9759dab1a8ae34d90512cfca64e2446dfbdac18875d7"}
Nov 04 02:20:12.992191 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.992021    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2" event={"ID":"746c1e17-96d9-40fd-8cce-8aebfbc17666","Type":"ContainerDied","Data":"55681a229eac391eb14fd425fa0e398e23befec27cbd9f2faf7fca686a23bb40"}
Nov 04 02:20:12.992191 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:12.992046    4865 scope.go:122] "RemoveContainer" containerID="a3f20a0df7b7913dbb4c9759dab1a8ae34d90512cfca64e2446dfbdac18875d7"
Nov 04 02:20:13.001871 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:13.001845    4865 scope.go:122] "RemoveContainer" containerID="a3f20a0df7b7913dbb4c9759dab1a8ae34d90512cfca64e2446dfbdac18875d7"
Nov 04 02:20:13.002150 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:20:13.002131    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3f20a0df7b7913dbb4c9759dab1a8ae34d90512cfca64e2446dfbdac18875d7\": container with ID starting with a3f20a0df7b7913dbb4c9759dab1a8ae34d90512cfca64e2446dfbdac18875d7 not found: ID does not exist" containerID="a3f20a0df7b7913dbb4c9759dab1a8ae34d90512cfca64e2446dfbdac18875d7"
Nov 04 02:20:13.002209 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:13.002161    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3f20a0df7b7913dbb4c9759dab1a8ae34d90512cfca64e2446dfbdac18875d7"} err="failed to get container status \"a3f20a0df7b7913dbb4c9759dab1a8ae34d90512cfca64e2446dfbdac18875d7\": rpc error: code = NotFound desc = could not find container \"a3f20a0df7b7913dbb4c9759dab1a8ae34d90512cfca64e2446dfbdac18875d7\": container with ID starting with a3f20a0df7b7913dbb4c9759dab1a8ae34d90512cfca64e2446dfbdac18875d7 not found: ID does not exist"
Nov 04 02:20:13.161149 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:13.161058    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"]
Nov 04 02:20:13.211485 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:13.211457    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2"]
Nov 04 02:20:14.895810 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:14.895732    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="746c1e17-96d9-40fd-8cce-8aebfbc17666" path="/var/lib/kubelet/pods/746c1e17-96d9-40fd-8cce-8aebfbc17666/volumes"
Nov 04 02:20:20.982361 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:20.982327    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:20:22.742217 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:22.742175    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"]
Nov 04 02:20:27.743787 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:27.743715    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6" podUID="959ef67d-bb84-4062-aff2-8f69275265b8" containerName="payload-processing" containerID="cri-o://fbbaeefd0d163583b296f3949351a7796364b42578c8759f490c92a2a1150cab" gracePeriod=25
Nov 04 02:20:28.013785 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.013756    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:20:28.057758 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.057714    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/959ef67d-bb84-4062-aff2-8f69275265b8-provider-credentials\" (UniqueName: \"kubernetes.io/projected/959ef67d-bb84-4062-aff2-8f69275265b8-provider-credentials\") pod \"959ef67d-bb84-4062-aff2-8f69275265b8\" (UID: \"959ef67d-bb84-4062-aff2-8f69275265b8\") "
Nov 04 02:20:28.057959 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.057817    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/959ef67d-bb84-4062-aff2-8f69275265b8-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/959ef67d-bb84-4062-aff2-8f69275265b8-routing-overlay-volume\") pod \"959ef67d-bb84-4062-aff2-8f69275265b8\" (UID: \"959ef67d-bb84-4062-aff2-8f69275265b8\") "
Nov 04 02:20:28.057959 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.057912    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/959ef67d-bb84-4062-aff2-8f69275265b8-kube-api-access-hb5s2\" (UniqueName: \"kubernetes.io/projected/959ef67d-bb84-4062-aff2-8f69275265b8-kube-api-access-hb5s2\") pod \"959ef67d-bb84-4062-aff2-8f69275265b8\" (UID: \"959ef67d-bb84-4062-aff2-8f69275265b8\") "
Nov 04 02:20:28.058072 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.057965    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/959ef67d-bb84-4062-aff2-8f69275265b8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/959ef67d-bb84-4062-aff2-8f69275265b8-plugins-config-volume\") pod \"959ef67d-bb84-4062-aff2-8f69275265b8\" (UID: \"959ef67d-bb84-4062-aff2-8f69275265b8\") "
Nov 04 02:20:28.058263 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.058233    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/959ef67d-bb84-4062-aff2-8f69275265b8-routing-overlay-volume" pod "959ef67d-bb84-4062-aff2-8f69275265b8" (UID: "959ef67d-bb84-4062-aff2-8f69275265b8"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:20:28.058429 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.058396    4865 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/959ef67d-bb84-4062-aff2-8f69275265b8-routing-overlay-volume\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:28.058492 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.058428    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/959ef67d-bb84-4062-aff2-8f69275265b8-plugins-config-volume" pod "959ef67d-bb84-4062-aff2-8f69275265b8" (UID: "959ef67d-bb84-4062-aff2-8f69275265b8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:20:28.060006 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.059981    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/959ef67d-bb84-4062-aff2-8f69275265b8-provider-credentials" pod "959ef67d-bb84-4062-aff2-8f69275265b8" (UID: "959ef67d-bb84-4062-aff2-8f69275265b8"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:20:28.060092 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.060071    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/959ef67d-bb84-4062-aff2-8f69275265b8-kube-api-access-hb5s2" pod "959ef67d-bb84-4062-aff2-8f69275265b8" (UID: "959ef67d-bb84-4062-aff2-8f69275265b8"). InnerVolumeSpecName "kube-api-access-hb5s2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:20:28.086520 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.086483    4865 generic.go:356] "Generic (PLEG): container finished" podID="959ef67d-bb84-4062-aff2-8f69275265b8" containerID="fbbaeefd0d163583b296f3949351a7796364b42578c8759f490c92a2a1150cab" exitCode=0
Nov 04 02:20:28.086689 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.086535    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"
Nov 04 02:20:28.086689 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.086564    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6" event={"ID":"959ef67d-bb84-4062-aff2-8f69275265b8","Type":"ContainerDied","Data":"fbbaeefd0d163583b296f3949351a7796364b42578c8759f490c92a2a1150cab"}
Nov 04 02:20:28.086689 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.086622    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6" event={"ID":"959ef67d-bb84-4062-aff2-8f69275265b8","Type":"ContainerDied","Data":"566b8e28c6a3cff18e042d8964a1289fc695ae05c4e971c475102aced8bb45f2"}
Nov 04 02:20:28.086689 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.086654    4865 scope.go:122] "RemoveContainer" containerID="fbbaeefd0d163583b296f3949351a7796364b42578c8759f490c92a2a1150cab"
Nov 04 02:20:28.096661 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.096639    4865 scope.go:122] "RemoveContainer" containerID="fbbaeefd0d163583b296f3949351a7796364b42578c8759f490c92a2a1150cab"
Nov 04 02:20:28.096961 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:20:28.096941    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbbaeefd0d163583b296f3949351a7796364b42578c8759f490c92a2a1150cab\": container with ID starting with fbbaeefd0d163583b296f3949351a7796364b42578c8759f490c92a2a1150cab not found: ID does not exist" containerID="fbbaeefd0d163583b296f3949351a7796364b42578c8759f490c92a2a1150cab"
Nov 04 02:20:28.097007 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.096972    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbbaeefd0d163583b296f3949351a7796364b42578c8759f490c92a2a1150cab"} err="failed to get container status \"fbbaeefd0d163583b296f3949351a7796364b42578c8759f490c92a2a1150cab\": rpc error: code = NotFound desc = could not find container \"fbbaeefd0d163583b296f3949351a7796364b42578c8759f490c92a2a1150cab\": container with ID starting with fbbaeefd0d163583b296f3949351a7796364b42578c8759f490c92a2a1150cab not found: ID does not exist"
Nov 04 02:20:28.159766 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.159719    4865 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/959ef67d-bb84-4062-aff2-8f69275265b8-plugins-config-volume\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:28.159766 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.159758    4865 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/959ef67d-bb84-4062-aff2-8f69275265b8-provider-credentials\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:28.159978 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.159794    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hb5s2\" (UniqueName: \"kubernetes.io/projected/959ef67d-bb84-4062-aff2-8f69275265b8-kube-api-access-hb5s2\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:28.247573 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.247481    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"]
Nov 04 02:20:28.256473 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.256438    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-3f76d0/payload-processing-external-model-e2e-extmod-b-3f76d0-5cffw9pc6"]
Nov 04 02:20:28.405707 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:20:28.405673    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:20:28.895616 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:28.895582    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="959ef67d-bb84-4062-aff2-8f69275265b8" path="/var/lib/kubelet/pods/959ef67d-bb84-4062-aff2-8f69275265b8/volumes"
Nov 04 02:20:34.516142 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:34.516110    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:20:37.047621 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:37.047590    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/2.log"
Nov 04 02:20:37.054985 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:37.054956    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/2.log"
Nov 04 02:20:56.550123 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:56.550084    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"]
Nov 04 02:20:56.550655 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:56.550368    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql" podUID="91d9868b-f91c-424b-8671-f2973afdfbfa" containerName="istio-proxy" containerID="cri-o://2c0a1f07f8442adb67dbf1ba8915f64876e27b2d2ede981f13e401b2f255fc9c" gracePeriod=30
Nov 04 02:20:58.360004 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:58.359969    4865 patch_prober.go:29] interesting pod/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 02:20:58.360406 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:20:58.360035    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql" podUID="91d9868b-f91c-424b-8671-f2973afdfbfa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 02:20:58.404694 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:20:58.404650    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:21:00.416857 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:00.416814    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:21:01.811833 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.811760    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:21:01.893097 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.893065    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91d9868b-f91c-424b-8671-f2973afdfbfa-kube-api-access-cqdcg\" (UniqueName: \"kubernetes.io/projected/91d9868b-f91c-424b-8671-f2973afdfbfa-kube-api-access-cqdcg\") pod \"91d9868b-f91c-424b-8671-f2973afdfbfa\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") "
Nov 04 02:21:01.893325 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.893133    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-envoy\") pod \"91d9868b-f91c-424b-8671-f2973afdfbfa\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") "
Nov 04 02:21:01.893325 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.893236    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-workload-socket\") pod \"91d9868b-f91c-424b-8671-f2973afdfbfa\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") "
Nov 04 02:21:01.893325 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.893311    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91d9868b-f91c-424b-8671-f2973afdfbfa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91d9868b-f91c-424b-8671-f2973afdfbfa-istiod-ca-cert\") pod \"91d9868b-f91c-424b-8671-f2973afdfbfa\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") "
Nov 04 02:21:01.893506 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.893351    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-credential-socket\") pod \"91d9868b-f91c-424b-8671-f2973afdfbfa\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") "
Nov 04 02:21:01.893506 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.893434    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-workload-certs\") pod \"91d9868b-f91c-424b-8671-f2973afdfbfa\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") "
Nov 04 02:21:01.893506 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.893498    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-data\") pod \"91d9868b-f91c-424b-8671-f2973afdfbfa\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") "
Nov 04 02:21:01.893661 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.893503    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-workload-socket" pod "91d9868b-f91c-424b-8671-f2973afdfbfa" (UID: "91d9868b-f91c-424b-8671-f2973afdfbfa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:01.893661 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.893565    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-podinfo\") pod \"91d9868b-f91c-424b-8671-f2973afdfbfa\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") "
Nov 04 02:21:01.893780 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.893657    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-token\" (UniqueName: \"kubernetes.io/projected/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-token\") pod \"91d9868b-f91c-424b-8671-f2973afdfbfa\" (UID: \"91d9868b-f91c-424b-8671-f2973afdfbfa\") "
Nov 04 02:21:01.893780 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.893717    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-workload-certs" pod "91d9868b-f91c-424b-8671-f2973afdfbfa" (UID: "91d9868b-f91c-424b-8671-f2973afdfbfa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:01.893780 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.893763    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-data" pod "91d9868b-f91c-424b-8671-f2973afdfbfa" (UID: "91d9868b-f91c-424b-8671-f2973afdfbfa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:01.893920 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.893692    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91d9868b-f91c-424b-8671-f2973afdfbfa-istiod-ca-cert" pod "91d9868b-f91c-424b-8671-f2973afdfbfa" (UID: "91d9868b-f91c-424b-8671-f2973afdfbfa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:01.893920 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.893869    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-credential-socket" pod "91d9868b-f91c-424b-8671-f2973afdfbfa" (UID: "91d9868b-f91c-424b-8671-f2973afdfbfa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:01.894117 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.894091    4865 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-workload-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:01.894192 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.894122    4865 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91d9868b-f91c-424b-8671-f2973afdfbfa-istiod-ca-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:01.894192 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.894138    4865 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-credential-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:01.894192 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.894153    4865 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-workload-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:01.894192 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.894168    4865 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-data\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:01.895463 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.895434    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91d9868b-f91c-424b-8671-f2973afdfbfa-kube-api-access-cqdcg" pod "91d9868b-f91c-424b-8671-f2973afdfbfa" (UID: "91d9868b-f91c-424b-8671-f2973afdfbfa"). InnerVolumeSpecName "kube-api-access-cqdcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:01.895628 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.895609    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-podinfo" pod "91d9868b-f91c-424b-8671-f2973afdfbfa" (UID: "91d9868b-f91c-424b-8671-f2973afdfbfa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:21:01.895805 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.895783    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-token" pod "91d9868b-f91c-424b-8671-f2973afdfbfa" (UID: "91d9868b-f91c-424b-8671-f2973afdfbfa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:01.895847 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.895783    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-envoy" pod "91d9868b-f91c-424b-8671-f2973afdfbfa" (UID: "91d9868b-f91c-424b-8671-f2973afdfbfa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:01.995764 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.995723    4865 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-envoy\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:01.995764 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.995764    4865 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-podinfo\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:01.996016 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.995778    4865 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91d9868b-f91c-424b-8671-f2973afdfbfa-istio-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:01.996016 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:01.995795    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqdcg\" (UniqueName: \"kubernetes.io/projected/91d9868b-f91c-424b-8671-f2973afdfbfa-kube-api-access-cqdcg\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:02.307575 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:02.307540    4865 generic.go:356] "Generic (PLEG): container finished" podID="91d9868b-f91c-424b-8671-f2973afdfbfa" containerID="2c0a1f07f8442adb67dbf1ba8915f64876e27b2d2ede981f13e401b2f255fc9c" exitCode=0
Nov 04 02:21:02.307575 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:02.307580    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql" event={"ID":"91d9868b-f91c-424b-8671-f2973afdfbfa","Type":"ContainerDied","Data":"2c0a1f07f8442adb67dbf1ba8915f64876e27b2d2ede981f13e401b2f255fc9c"}
Nov 04 02:21:02.307827 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:02.307597    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"
Nov 04 02:21:02.307827 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:02.307636    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql" event={"ID":"91d9868b-f91c-424b-8671-f2973afdfbfa","Type":"ContainerDied","Data":"e43ee408cb22466d5fa7a3292c0fcc7d0855cbd95f253615b6539b683582ced7"}
Nov 04 02:21:02.307827 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:02.307658    4865 scope.go:122] "RemoveContainer" containerID="2c0a1f07f8442adb67dbf1ba8915f64876e27b2d2ede981f13e401b2f255fc9c"
Nov 04 02:21:02.317583 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:02.317559    4865 scope.go:122] "RemoveContainer" containerID="938e6fe08c4824de9f1c46784d8e7b6509d0d3e2f3052fdb91bc1f8f601c09c3"
Nov 04 02:21:02.326680 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:02.326660    4865 scope.go:122] "RemoveContainer" containerID="2c0a1f07f8442adb67dbf1ba8915f64876e27b2d2ede981f13e401b2f255fc9c"
Nov 04 02:21:02.327026 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:21:02.326997    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c0a1f07f8442adb67dbf1ba8915f64876e27b2d2ede981f13e401b2f255fc9c\": container with ID starting with 2c0a1f07f8442adb67dbf1ba8915f64876e27b2d2ede981f13e401b2f255fc9c not found: ID does not exist" containerID="2c0a1f07f8442adb67dbf1ba8915f64876e27b2d2ede981f13e401b2f255fc9c"
Nov 04 02:21:02.327086 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:02.327039    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c0a1f07f8442adb67dbf1ba8915f64876e27b2d2ede981f13e401b2f255fc9c"} err="failed to get container status \"2c0a1f07f8442adb67dbf1ba8915f64876e27b2d2ede981f13e401b2f255fc9c\": rpc error: code = NotFound desc = could not find container \"2c0a1f07f8442adb67dbf1ba8915f64876e27b2d2ede981f13e401b2f255fc9c\": container with ID starting with 2c0a1f07f8442adb67dbf1ba8915f64876e27b2d2ede981f13e401b2f255fc9c not found: ID does not exist"
Nov 04 02:21:02.327086 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:02.327064    4865 scope.go:122] "RemoveContainer" containerID="938e6fe08c4824de9f1c46784d8e7b6509d0d3e2f3052fdb91bc1f8f601c09c3"
Nov 04 02:21:02.327341 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:21:02.327320    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"938e6fe08c4824de9f1c46784d8e7b6509d0d3e2f3052fdb91bc1f8f601c09c3\": container with ID starting with 938e6fe08c4824de9f1c46784d8e7b6509d0d3e2f3052fdb91bc1f8f601c09c3 not found: ID does not exist" containerID="938e6fe08c4824de9f1c46784d8e7b6509d0d3e2f3052fdb91bc1f8f601c09c3"
Nov 04 02:21:02.327438 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:02.327344    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"938e6fe08c4824de9f1c46784d8e7b6509d0d3e2f3052fdb91bc1f8f601c09c3"} err="failed to get container status \"938e6fe08c4824de9f1c46784d8e7b6509d0d3e2f3052fdb91bc1f8f601c09c3\": rpc error: code = NotFound desc = could not find container \"938e6fe08c4824de9f1c46784d8e7b6509d0d3e2f3052fdb91bc1f8f601c09c3\": container with ID starting with 938e6fe08c4824de9f1c46784d8e7b6509d0d3e2f3052fdb91bc1f8f601c09c3 not found: ID does not exist"
Nov 04 02:21:02.475904 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:02.475873    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"]
Nov 04 02:21:02.481858 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:02.481835    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql"]
Nov 04 02:21:02.896330 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:02.896276    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91d9868b-f91c-424b-8671-f2973afdfbfa" path="/var/lib/kubelet/pods/91d9868b-f91c-424b-8671-f2973afdfbfa/volumes"
Nov 04 02:21:17.494235 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:17.494176    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"]
Nov 04 02:21:22.498873 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:22.498814    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb" podUID="3a034e7b-5c1c-49b8-a850-cac01d4ec932" containerName="payload-processing" containerID="cri-o://4b32c44132dab2c482463f94c8293bc4029c4901c6efdc42fbe12ebbf768c61e" gracePeriod=25
Nov 04 02:21:22.754296 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:22.754226    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:21:22.764434 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:22.764413    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a034e7b-5c1c-49b8-a850-cac01d4ec932-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a034e7b-5c1c-49b8-a850-cac01d4ec932-plugins-config-volume\") pod \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\" (UID: \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\") "
Nov 04 02:21:22.764529 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:22.764461    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a034e7b-5c1c-49b8-a850-cac01d4ec932-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/3a034e7b-5c1c-49b8-a850-cac01d4ec932-routing-overlay-volume\") pod \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\" (UID: \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\") "
Nov 04 02:21:22.764529 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:22.764500    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a034e7b-5c1c-49b8-a850-cac01d4ec932-kube-api-access-rt9xj\" (UniqueName: \"kubernetes.io/projected/3a034e7b-5c1c-49b8-a850-cac01d4ec932-kube-api-access-rt9xj\") pod \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\" (UID: \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\") "
Nov 04 02:21:22.764618 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:22.764585    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a034e7b-5c1c-49b8-a850-cac01d4ec932-provider-credentials\" (UniqueName: \"kubernetes.io/projected/3a034e7b-5c1c-49b8-a850-cac01d4ec932-provider-credentials\") pod \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\" (UID: \"3a034e7b-5c1c-49b8-a850-cac01d4ec932\") "
Nov 04 02:21:22.764804 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:22.764780    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a034e7b-5c1c-49b8-a850-cac01d4ec932-plugins-config-volume" pod "3a034e7b-5c1c-49b8-a850-cac01d4ec932" (UID: "3a034e7b-5c1c-49b8-a850-cac01d4ec932"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:22.764877 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:22.764828    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a034e7b-5c1c-49b8-a850-cac01d4ec932-routing-overlay-volume" pod "3a034e7b-5c1c-49b8-a850-cac01d4ec932" (UID: "3a034e7b-5c1c-49b8-a850-cac01d4ec932"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:22.765014 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:22.764993    4865 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a034e7b-5c1c-49b8-a850-cac01d4ec932-plugins-config-volume\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:22.765014 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:22.765013    4865 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/3a034e7b-5c1c-49b8-a850-cac01d4ec932-routing-overlay-volume\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:22.766703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:22.766683    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a034e7b-5c1c-49b8-a850-cac01d4ec932-provider-credentials" pod "3a034e7b-5c1c-49b8-a850-cac01d4ec932" (UID: "3a034e7b-5c1c-49b8-a850-cac01d4ec932"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:22.766809 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:22.766788    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a034e7b-5c1c-49b8-a850-cac01d4ec932-kube-api-access-rt9xj" pod "3a034e7b-5c1c-49b8-a850-cac01d4ec932" (UID: "3a034e7b-5c1c-49b8-a850-cac01d4ec932"). InnerVolumeSpecName "kube-api-access-rt9xj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:22.865977 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:22.865947    4865 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/3a034e7b-5c1c-49b8-a850-cac01d4ec932-provider-credentials\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:22.865977 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:22.865973    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rt9xj\" (UniqueName: \"kubernetes.io/projected/3a034e7b-5c1c-49b8-a850-cac01d4ec932-kube-api-access-rt9xj\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:23.451842 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:23.451808    4865 generic.go:356] "Generic (PLEG): container finished" podID="3a034e7b-5c1c-49b8-a850-cac01d4ec932" containerID="4b32c44132dab2c482463f94c8293bc4029c4901c6efdc42fbe12ebbf768c61e" exitCode=0
Nov 04 02:21:23.452016 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:23.451854    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"
Nov 04 02:21:23.452016 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:23.451858    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb" event={"ID":"3a034e7b-5c1c-49b8-a850-cac01d4ec932","Type":"ContainerDied","Data":"4b32c44132dab2c482463f94c8293bc4029c4901c6efdc42fbe12ebbf768c61e"}
Nov 04 02:21:23.452016 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:23.451973    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb" event={"ID":"3a034e7b-5c1c-49b8-a850-cac01d4ec932","Type":"ContainerDied","Data":"b6b398315a5690f4f720c67fc9d12cb6ad7ee25794c071052d365a7cfe971ff2"}
Nov 04 02:21:23.452016 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:23.452008    4865 scope.go:122] "RemoveContainer" containerID="4b32c44132dab2c482463f94c8293bc4029c4901c6efdc42fbe12ebbf768c61e"
Nov 04 02:21:23.460899 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:23.460881    4865 scope.go:122] "RemoveContainer" containerID="4b32c44132dab2c482463f94c8293bc4029c4901c6efdc42fbe12ebbf768c61e"
Nov 04 02:21:23.461150 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:21:23.461131    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b32c44132dab2c482463f94c8293bc4029c4901c6efdc42fbe12ebbf768c61e\": container with ID starting with 4b32c44132dab2c482463f94c8293bc4029c4901c6efdc42fbe12ebbf768c61e not found: ID does not exist" containerID="4b32c44132dab2c482463f94c8293bc4029c4901c6efdc42fbe12ebbf768c61e"
Nov 04 02:21:23.461188 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:23.461161    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b32c44132dab2c482463f94c8293bc4029c4901c6efdc42fbe12ebbf768c61e"} err="failed to get container status \"4b32c44132dab2c482463f94c8293bc4029c4901c6efdc42fbe12ebbf768c61e\": rpc error: code = NotFound desc = could not find container \"4b32c44132dab2c482463f94c8293bc4029c4901c6efdc42fbe12ebbf768c61e\": container with ID starting with 4b32c44132dab2c482463f94c8293bc4029c4901c6efdc42fbe12ebbf768c61e not found: ID does not exist"
Nov 04 02:21:23.618884 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:23.618854    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"]
Nov 04 02:21:23.622904 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:23.622881    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-3d082d/payload-processing-external-model-e2e-extmod-b-3d082d-fcb8lvzkb"]
Nov 04 02:21:24.895840 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:24.895805    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a034e7b-5c1c-49b8-a850-cac01d4ec932" path="/var/lib/kubelet/pods/3a034e7b-5c1c-49b8-a850-cac01d4ec932/volumes"
Nov 04 02:21:27.227853 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:27.227820    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:21:28.404731 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:21:28.404692    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:21:29.197152 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.197116    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"]
Nov 04 02:21:29.197357 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.197334    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r" podUID="6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb" containerName="maas-api" containerID="cri-o://ab64282eca5aeb206849dc6971fb26abf3ff4088e38b6cd46ce8c515ccc3f70b" gracePeriod=30
Nov 04 02:21:29.459406 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.459312    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:21:29.502358 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.495773    4865 generic.go:356] "Generic (PLEG): container finished" podID="6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb" containerID="ab64282eca5aeb206849dc6971fb26abf3ff4088e38b6cd46ce8c515ccc3f70b" exitCode=0
Nov 04 02:21:29.502358 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.495879    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r" event={"ID":"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb","Type":"ContainerDied","Data":"ab64282eca5aeb206849dc6971fb26abf3ff4088e38b6cd46ce8c515ccc3f70b"}
Nov 04 02:21:29.502358 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.495935    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r" event={"ID":"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb","Type":"ContainerDied","Data":"be23f4836cf1a0c0291e99ad95a8b62c64538d23be6279478180ffa476f250f2"}
Nov 04 02:21:29.502358 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.495966    4865 scope.go:122] "RemoveContainer" containerID="ab64282eca5aeb206849dc6971fb26abf3ff4088e38b6cd46ce8c515ccc3f70b"
Nov 04 02:21:29.502358 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.496125    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"
Nov 04 02:21:29.532577 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.532553    4865 scope.go:122] "RemoveContainer" containerID="ab64282eca5aeb206849dc6971fb26abf3ff4088e38b6cd46ce8c515ccc3f70b"
Nov 04 02:21:29.532944 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:21:29.532917    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab64282eca5aeb206849dc6971fb26abf3ff4088e38b6cd46ce8c515ccc3f70b\": container with ID starting with ab64282eca5aeb206849dc6971fb26abf3ff4088e38b6cd46ce8c515ccc3f70b not found: ID does not exist" containerID="ab64282eca5aeb206849dc6971fb26abf3ff4088e38b6cd46ce8c515ccc3f70b"
Nov 04 02:21:29.533024 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.532954    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab64282eca5aeb206849dc6971fb26abf3ff4088e38b6cd46ce8c515ccc3f70b"} err="failed to get container status \"ab64282eca5aeb206849dc6971fb26abf3ff4088e38b6cd46ce8c515ccc3f70b\": rpc error: code = NotFound desc = could not find container \"ab64282eca5aeb206849dc6971fb26abf3ff4088e38b6cd46ce8c515ccc3f70b\": container with ID starting with ab64282eca5aeb206849dc6971fb26abf3ff4088e38b6cd46ce8c515ccc3f70b not found: ID does not exist"
Nov 04 02:21:29.537313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.537262    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-maas-api-tls\") pod \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\" (UID: \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\") "
Nov 04 02:21:29.537396 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.537312    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-kube-api-access-d8kb7\" (UniqueName: \"kubernetes.io/projected/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-kube-api-access-d8kb7\") pod \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\" (UID: \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\") "
Nov 04 02:21:29.537396 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.537346    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-metrics-certs\") pod \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\" (UID: \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\") "
Nov 04 02:21:29.537602 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.537583    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-openshift-service-ca\") pod \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\" (UID: \"6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb\") "
Nov 04 02:21:29.537996 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.537968    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-openshift-service-ca" pod "6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb" (UID: "6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:29.539465 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.539427    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-metrics-certs" pod "6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb" (UID: "6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:21:29.539559 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.539465    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-kube-api-access-d8kb7" pod "6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb" (UID: "6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb"). InnerVolumeSpecName "kube-api-access-d8kb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:29.539559 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.539482    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-maas-api-tls" pod "6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb" (UID: "6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:21:29.661322 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.657683    4865 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-openshift-service-ca\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:29.661322 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.657730    4865 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-maas-api-tls\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:29.661322 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.657763    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8kb7\" (UniqueName: \"kubernetes.io/projected/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-kube-api-access-d8kb7\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:29.661322 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.657783    4865 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb-metrics-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:29.920950 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.920910    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"]
Nov 04 02:21:29.924866 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:29.924841    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-3f76d0-65f676cf6b-jw45r"]
Nov 04 02:21:30.895880 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:30.895847    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb" path="/var/lib/kubelet/pods/6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb/volumes"
Nov 04 02:21:33.637039 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.637005    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"]
Nov 04 02:21:33.637509 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.637494    4865 cpu_manager.go:414] "containerMap: removing container" podUID="91d9868b-f91c-424b-8671-f2973afdfbfa" containerName="istio-proxy"
Nov 04 02:21:33.637558 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.637510    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91d9868b-f91c-424b-8671-f2973afdfbfa" containerName="istio-proxy"
Nov 04 02:21:33.637558 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.637526    4865 cpu_manager.go:414] "containerMap: removing container" podUID="6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb" containerName="maas-api"
Nov 04 02:21:33.637558 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.637531    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb" containerName="maas-api"
Nov 04 02:21:33.637558 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.637551    4865 cpu_manager.go:414] "containerMap: removing container" podUID="3a034e7b-5c1c-49b8-a850-cac01d4ec932" containerName="payload-processing"
Nov 04 02:21:33.637558 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.637557    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a034e7b-5c1c-49b8-a850-cac01d4ec932" containerName="payload-processing"
Nov 04 02:21:33.637703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.637567    4865 cpu_manager.go:414] "containerMap: removing container" podUID="959ef67d-bb84-4062-aff2-8f69275265b8" containerName="payload-processing"
Nov 04 02:21:33.637703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.637572    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="959ef67d-bb84-4062-aff2-8f69275265b8" containerName="payload-processing"
Nov 04 02:21:33.637703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.637587    4865 cpu_manager.go:414] "containerMap: removing container" podUID="746c1e17-96d9-40fd-8cce-8aebfbc17666" containerName="istio-proxy"
Nov 04 02:21:33.637703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.637591    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="746c1e17-96d9-40fd-8cce-8aebfbc17666" containerName="istio-proxy"
Nov 04 02:21:33.637703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.637663    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="91d9868b-f91c-424b-8671-f2973afdfbfa" containerName="istio-proxy"
Nov 04 02:21:33.637703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.637669    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="746c1e17-96d9-40fd-8cce-8aebfbc17666" containerName="istio-proxy"
Nov 04 02:21:33.637703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.637675    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c7a83a4-3356-417d-ad04-f258a97655fd" containerName="istio-proxy"
Nov 04 02:21:33.637703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.637685    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="6dc8d5b0-f8df-405b-a02d-3ca353f7ecbb" containerName="maas-api"
Nov 04 02:21:33.637703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.637693    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a034e7b-5c1c-49b8-a850-cac01d4ec932" containerName="payload-processing"
Nov 04 02:21:33.637703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.637699    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="959ef67d-bb84-4062-aff2-8f69275265b8" containerName="payload-processing"
Nov 04 02:21:33.642116 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.642099    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:21:33.730032 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.729998    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq7rg\" (UniqueName: \"kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-kube-api-access-tq7rg\") pod \"payload-processing-external-model-5f69dbdb4d-vvrwp\" (UID: \"742654db-eedb-4c8d-b61a-504ae2932fea\") " pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:21:33.730223 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.730184    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/742654db-eedb-4c8d-b61a-504ae2932fea-plugins-config-volume\") pod \"payload-processing-external-model-5f69dbdb4d-vvrwp\" (UID: \"742654db-eedb-4c8d-b61a-504ae2932fea\") " pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:21:33.730314 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.730241    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/742654db-eedb-4c8d-b61a-504ae2932fea-routing-overlay-volume\") pod \"payload-processing-external-model-5f69dbdb4d-vvrwp\" (UID: \"742654db-eedb-4c8d-b61a-504ae2932fea\") " pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:21:33.730412 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.730381    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials\") pod \"payload-processing-external-model-5f69dbdb4d-vvrwp\" (UID: \"742654db-eedb-4c8d-b61a-504ae2932fea\") " pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:21:33.746405 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.746379    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Nov 04 02:21:33.746405 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.746400    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-5wqjh\""
Nov 04 02:21:33.746405 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.746387    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Nov 04 02:21:33.746612 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.746403    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-ee7ade4978e6\""
Nov 04 02:21:33.746612 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.746403    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-3ebb9bb78423\""
Nov 04 02:21:33.746612 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.746434    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Nov 04 02:21:33.746612 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.746516    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Nov 04 02:21:33.748915 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.748896    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"]
Nov 04 02:21:33.831145 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.831119    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/742654db-eedb-4c8d-b61a-504ae2932fea-routing-overlay-volume\") pod \"payload-processing-external-model-5f69dbdb4d-vvrwp\" (UID: \"742654db-eedb-4c8d-b61a-504ae2932fea\") " pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:21:33.831299 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.831163    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials\") pod \"payload-processing-external-model-5f69dbdb4d-vvrwp\" (UID: \"742654db-eedb-4c8d-b61a-504ae2932fea\") " pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:21:33.831299 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.831203    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq7rg\" (UniqueName: \"kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-kube-api-access-tq7rg\") pod \"payload-processing-external-model-5f69dbdb4d-vvrwp\" (UID: \"742654db-eedb-4c8d-b61a-504ae2932fea\") " pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:21:33.831418 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.831359    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/742654db-eedb-4c8d-b61a-504ae2932fea-plugins-config-volume\") pod \"payload-processing-external-model-5f69dbdb4d-vvrwp\" (UID: \"742654db-eedb-4c8d-b61a-504ae2932fea\") " pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:21:33.831948 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.831924    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/742654db-eedb-4c8d-b61a-504ae2932fea-routing-overlay-volume\") pod \"payload-processing-external-model-5f69dbdb4d-vvrwp\" (UID: \"742654db-eedb-4c8d-b61a-504ae2932fea\") " pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:21:33.832108 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.832093    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/742654db-eedb-4c8d-b61a-504ae2932fea-plugins-config-volume\") pod \"payload-processing-external-model-5f69dbdb4d-vvrwp\" (UID: \"742654db-eedb-4c8d-b61a-504ae2932fea\") " pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:21:33.833715 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.833697    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials\") pod \"payload-processing-external-model-5f69dbdb4d-vvrwp\" (UID: \"742654db-eedb-4c8d-b61a-504ae2932fea\") " pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:21:33.893321 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.893239    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq7rg\" (UniqueName: \"kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-kube-api-access-tq7rg\") pod \"payload-processing-external-model-5f69dbdb4d-vvrwp\" (UID: \"742654db-eedb-4c8d-b61a-504ae2932fea\") " pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:21:33.955075 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:33.955050    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:21:34.331479 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:34.331441    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"]
Nov 04 02:21:34.332667 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:21:34.332640    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod742654db_eedb_4c8d_b61a_504ae2932fea.slice/crio-481b7d89f2d65c1c67630cc42c02347cb0067c16bf2b467935bb7bff33b7fce2 WatchSource:0}: Error finding container 481b7d89f2d65c1c67630cc42c02347cb0067c16bf2b467935bb7bff33b7fce2: Status 404 returned error can't find the container with id 481b7d89f2d65c1c67630cc42c02347cb0067c16bf2b467935bb7bff33b7fce2
Nov 04 02:21:34.525710 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:34.525673    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp" event={"ID":"742654db-eedb-4c8d-b61a-504ae2932fea","Type":"ContainerStarted","Data":"3b2bfc056ff4b0e23ad26ca200f6398d5cd15fc7398081328fdb6e4e19099887"}
Nov 04 02:21:34.525883 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:34.525734    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp" event={"ID":"742654db-eedb-4c8d-b61a-504ae2932fea","Type":"ContainerStarted","Data":"481b7d89f2d65c1c67630cc42c02347cb0067c16bf2b467935bb7bff33b7fce2"}
Nov 04 02:21:34.525936 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:34.525884    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:21:34.645970 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:34.645883    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp" podStartSLOduration=1.645870556 podStartE2EDuration="1.645870556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:21:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:21:34.643744698 +0000 UTC m=+1557.870531368" watchObservedRunningTime="2026-11-04 02:21:34.645870556 +0000 UTC m=+1557.872657280"
Nov 04 02:21:40.907505 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:40.907471    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"]
Nov 04 02:21:40.958049 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:40.958021    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"]
Nov 04 02:21:45.534962 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:45.534881    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:21:55.927770 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:55.927728    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx" podUID="5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1" containerName="main" containerID="cri-o://6d74aa9f1d49564a934082f6a44c6c2435ff923e3bd924d885f5dc5331684e70" gracePeriod=45
Nov 04 02:21:55.977054 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:55.977023    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4" podUID="43d28dad-923c-40d5-96e9-10681d488d0b" containerName="main" containerID="cri-o://a683862d66fd964399d40af78f1c8de94c89b13346d55a62ba54cb57e9a81dc9" gracePeriod=45
Nov 04 02:21:55.982619 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:55.982591    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx" podUID="5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": dial tcp 10.132.0.57:8000: connect: connection refused"
Nov 04 02:21:56.266862 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.266823    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:21:56.277804 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.277782    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:21:56.336830 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.336795    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-home\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-home\") pod \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") "
Nov 04 02:21:56.337011 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.336885    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-dshm\") pod \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") "
Nov 04 02:21:56.337011 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.336924    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-model-cache\") pod \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") "
Nov 04 02:21:56.337011 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.336964    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-kube-api-access-bs8jb\" (UniqueName: \"kubernetes.io/projected/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-kube-api-access-bs8jb\") pod \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") "
Nov 04 02:21:56.337011 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.337004    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-tmp-dir\") pod \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") "
Nov 04 02:21:56.337251 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.337111    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-tls-certs\" (UniqueName: \"kubernetes.io/secret/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-tls-certs\") pod \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\" (UID: \"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1\") "
Nov 04 02:21:56.338244 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.338200    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-model-cache" pod "5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1" (UID: "5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:56.338244 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.338213    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-tmp-dir" pod "5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1" (UID: "5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:56.338494 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.338402    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-home" pod "5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1" (UID: "5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:56.340059 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.340020    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-tls-certs" pod "5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1" (UID: "5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:21:56.340183 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.340088    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-kube-api-access-bs8jb" pod "5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1" (UID: "5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1"). InnerVolumeSpecName "kube-api-access-bs8jb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:56.340244 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.340178    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-dshm" pod "5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1" (UID: "5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:56.438711 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.438675    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-model-cache\") pod \"43d28dad-923c-40d5-96e9-10681d488d0b\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") "
Nov 04 02:21:56.438884 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.438748    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43d28dad-923c-40d5-96e9-10681d488d0b-kube-api-access-pwjrt\" (UniqueName: \"kubernetes.io/projected/43d28dad-923c-40d5-96e9-10681d488d0b-kube-api-access-pwjrt\") pod \"43d28dad-923c-40d5-96e9-10681d488d0b\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") "
Nov 04 02:21:56.438884 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.438845    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-tmp-dir\") pod \"43d28dad-923c-40d5-96e9-10681d488d0b\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") "
Nov 04 02:21:56.439002 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.438885    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-dshm\") pod \"43d28dad-923c-40d5-96e9-10681d488d0b\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") "
Nov 04 02:21:56.439002 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.438935    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/43d28dad-923c-40d5-96e9-10681d488d0b-tls-certs\" (UniqueName: \"kubernetes.io/secret/43d28dad-923c-40d5-96e9-10681d488d0b-tls-certs\") pod \"43d28dad-923c-40d5-96e9-10681d488d0b\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") "
Nov 04 02:21:56.439002 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.438975    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-model-cache" pod "43d28dad-923c-40d5-96e9-10681d488d0b" (UID: "43d28dad-923c-40d5-96e9-10681d488d0b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:56.439165 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.439046    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-home\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-home\") pod \"43d28dad-923c-40d5-96e9-10681d488d0b\" (UID: \"43d28dad-923c-40d5-96e9-10681d488d0b\") "
Nov 04 02:21:56.439165 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.439132    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-tmp-dir" pod "43d28dad-923c-40d5-96e9-10681d488d0b" (UID: "43d28dad-923c-40d5-96e9-10681d488d0b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:56.439472 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.439446    4865 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-model-cache\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.439597 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.439476    4865 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-home\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.439597 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.439492    4865 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-dshm\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.439597 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.439504    4865 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-model-cache\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.439597 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.439515    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs8jb\" (UniqueName: \"kubernetes.io/projected/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-kube-api-access-bs8jb\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.439597 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.439450    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-home" pod "43d28dad-923c-40d5-96e9-10681d488d0b" (UID: "43d28dad-923c-40d5-96e9-10681d488d0b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:56.439597 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.439548    4865 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-tmp-dir\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.439597 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.439564    4865 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-tmp-dir\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.439597 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.439576    4865 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1-tls-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.440985 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.440960    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43d28dad-923c-40d5-96e9-10681d488d0b-tls-certs" pod "43d28dad-923c-40d5-96e9-10681d488d0b" (UID: "43d28dad-923c-40d5-96e9-10681d488d0b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:21:56.441080 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.441055    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-dshm" pod "43d28dad-923c-40d5-96e9-10681d488d0b" (UID: "43d28dad-923c-40d5-96e9-10681d488d0b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:56.441208 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.441192    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43d28dad-923c-40d5-96e9-10681d488d0b-kube-api-access-pwjrt" pod "43d28dad-923c-40d5-96e9-10681d488d0b" (UID: "43d28dad-923c-40d5-96e9-10681d488d0b"). InnerVolumeSpecName "kube-api-access-pwjrt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:56.540999 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.540909    4865 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-home\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.540999 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.540950    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwjrt\" (UniqueName: \"kubernetes.io/projected/43d28dad-923c-40d5-96e9-10681d488d0b-kube-api-access-pwjrt\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.540999 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.540966    4865 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43d28dad-923c-40d5-96e9-10681d488d0b-dshm\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.540999 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.540982    4865 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43d28dad-923c-40d5-96e9-10681d488d0b-tls-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.682979 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.682937    4865 generic.go:356] "Generic (PLEG): container finished" podID="5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1" containerID="6d74aa9f1d49564a934082f6a44c6c2435ff923e3bd924d885f5dc5331684e70" exitCode=0
Nov 04 02:21:56.683154 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.682993    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"
Nov 04 02:21:56.683154 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.683020    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx" event={"ID":"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1","Type":"ContainerDied","Data":"6d74aa9f1d49564a934082f6a44c6c2435ff923e3bd924d885f5dc5331684e70"}
Nov 04 02:21:56.683154 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.683071    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx" event={"ID":"5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1","Type":"ContainerDied","Data":"bc192d8070531b72274050a12e5a15a036a874b47d6f6dbe1b7df7146cae19e1"}
Nov 04 02:21:56.683154 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.683091    4865 scope.go:122] "RemoveContainer" containerID="6d74aa9f1d49564a934082f6a44c6c2435ff923e3bd924d885f5dc5331684e70"
Nov 04 02:21:56.684887 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.684863    4865 generic.go:356] "Generic (PLEG): container finished" podID="43d28dad-923c-40d5-96e9-10681d488d0b" containerID="a683862d66fd964399d40af78f1c8de94c89b13346d55a62ba54cb57e9a81dc9" exitCode=0
Nov 04 02:21:56.685019 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.684915    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4" event={"ID":"43d28dad-923c-40d5-96e9-10681d488d0b","Type":"ContainerDied","Data":"a683862d66fd964399d40af78f1c8de94c89b13346d55a62ba54cb57e9a81dc9"}
Nov 04 02:21:56.685019 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.684944    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4" event={"ID":"43d28dad-923c-40d5-96e9-10681d488d0b","Type":"ContainerDied","Data":"e00517f6e75fce2ae028d79f98c634f3d36adaa09ba650930a9d1df91afe7f6f"}
Nov 04 02:21:56.685019 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.685004    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"
Nov 04 02:21:56.694884 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.694862    4865 scope.go:122] "RemoveContainer" containerID="6d74aa9f1d49564a934082f6a44c6c2435ff923e3bd924d885f5dc5331684e70"
Nov 04 02:21:56.695252 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:21:56.695227    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d74aa9f1d49564a934082f6a44c6c2435ff923e3bd924d885f5dc5331684e70\": container with ID starting with 6d74aa9f1d49564a934082f6a44c6c2435ff923e3bd924d885f5dc5331684e70 not found: ID does not exist" containerID="6d74aa9f1d49564a934082f6a44c6c2435ff923e3bd924d885f5dc5331684e70"
Nov 04 02:21:56.695380 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.695260    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d74aa9f1d49564a934082f6a44c6c2435ff923e3bd924d885f5dc5331684e70"} err="failed to get container status \"6d74aa9f1d49564a934082f6a44c6c2435ff923e3bd924d885f5dc5331684e70\": rpc error: code = NotFound desc = could not find container \"6d74aa9f1d49564a934082f6a44c6c2435ff923e3bd924d885f5dc5331684e70\": container with ID starting with 6d74aa9f1d49564a934082f6a44c6c2435ff923e3bd924d885f5dc5331684e70 not found: ID does not exist"
Nov 04 02:21:56.695380 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.695310    4865 scope.go:122] "RemoveContainer" containerID="a683862d66fd964399d40af78f1c8de94c89b13346d55a62ba54cb57e9a81dc9"
Nov 04 02:21:56.703946 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.703925    4865 scope.go:122] "RemoveContainer" containerID="a683862d66fd964399d40af78f1c8de94c89b13346d55a62ba54cb57e9a81dc9"
Nov 04 02:21:56.704214 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:21:56.704195    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a683862d66fd964399d40af78f1c8de94c89b13346d55a62ba54cb57e9a81dc9\": container with ID starting with a683862d66fd964399d40af78f1c8de94c89b13346d55a62ba54cb57e9a81dc9 not found: ID does not exist" containerID="a683862d66fd964399d40af78f1c8de94c89b13346d55a62ba54cb57e9a81dc9"
Nov 04 02:21:56.704312 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.704225    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a683862d66fd964399d40af78f1c8de94c89b13346d55a62ba54cb57e9a81dc9"} err="failed to get container status \"a683862d66fd964399d40af78f1c8de94c89b13346d55a62ba54cb57e9a81dc9\": rpc error: code = NotFound desc = could not find container \"a683862d66fd964399d40af78f1c8de94c89b13346d55a62ba54cb57e9a81dc9\": container with ID starting with a683862d66fd964399d40af78f1c8de94c89b13346d55a62ba54cb57e9a81dc9 not found: ID does not exist"
Nov 04 02:21:56.851216 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.851135    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"]
Nov 04 02:21:56.857203 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.857172    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/premium-w0-24bea9-kserve-65b97b57bf-hvlc4"]
Nov 04 02:21:56.895691 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.895663    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="43d28dad-923c-40d5-96e9-10681d488d0b" path="/var/lib/kubelet/pods/43d28dad-923c-40d5-96e9-10681d488d0b/volumes"
Nov 04 02:21:56.968756 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.968725    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"]
Nov 04 02:21:56.975138 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:56.975107    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/distinct-w0-24bea9-kserve-5f7df6bc7b-xjlvx"]
Nov 04 02:21:58.404337 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:21:58.404303    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:21:58.895497 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:21:58.895463    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1" path="/var/lib/kubelet/pods/5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1/volumes"
Nov 04 02:22:00.544053 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:00.544018    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:22:04.527388 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:04.526424    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:22:04.527388 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:04.526501    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd" podUID="ffac90da-23d5-4cf1-9dcf-f543bc11736b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:22:04.663755 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:04.663401    4865 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:22:04.663755 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:04.663478    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw" podUID="15993036-6c89-4844-b870-ccb12cfc744a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:22:10.589100 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.589058    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-68df898747-shn7h"]
Nov 04 02:22:10.589650 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.589632    4865 cpu_manager.go:414] "containerMap: removing container" podUID="43d28dad-923c-40d5-96e9-10681d488d0b" containerName="main"
Nov 04 02:22:10.589697 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.589651    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="43d28dad-923c-40d5-96e9-10681d488d0b" containerName="main"
Nov 04 02:22:10.589697 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.589666    4865 cpu_manager.go:414] "containerMap: removing container" podUID="5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1" containerName="main"
Nov 04 02:22:10.589697 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.589673    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1" containerName="main"
Nov 04 02:22:10.589802 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.589698    4865 cpu_manager.go:414] "containerMap: removing container" podUID="91d9868b-f91c-424b-8671-f2973afdfbfa" containerName="istio-proxy"
Nov 04 02:22:10.589802 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.589705    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91d9868b-f91c-424b-8671-f2973afdfbfa" containerName="istio-proxy"
Nov 04 02:22:10.589868 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.589809    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="43d28dad-923c-40d5-96e9-10681d488d0b" containerName="main"
Nov 04 02:22:10.589868 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.589820    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ab402a2-4ffb-4a93-b02d-5b8d1bda18f1" containerName="main"
Nov 04 02:22:10.589868 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.589826    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="91d9868b-f91c-424b-8671-f2973afdfbfa" containerName="istio-proxy"
Nov 04 02:22:10.594589 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.594572    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h"
Nov 04 02:22:10.690898 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.690865    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/344725fe-7870-483c-b76b-74f4069f3386-routing-overlay-volume\") pod \"payload-processing-external-model-68df898747-shn7h\" (UID: \"344725fe-7870-483c-b76b-74f4069f3386\") " pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h"
Nov 04 02:22:10.691051 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.690982    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/344725fe-7870-483c-b76b-74f4069f3386-plugins-config-volume\") pod \"payload-processing-external-model-68df898747-shn7h\" (UID: \"344725fe-7870-483c-b76b-74f4069f3386\") " pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h"
Nov 04 02:22:10.691051 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.691029    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/344725fe-7870-483c-b76b-74f4069f3386-provider-credentials\") pod \"payload-processing-external-model-68df898747-shn7h\" (UID: \"344725fe-7870-483c-b76b-74f4069f3386\") " pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h"
Nov 04 02:22:10.691158 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.691053    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q794r\" (UniqueName: \"kubernetes.io/projected/344725fe-7870-483c-b76b-74f4069f3386-kube-api-access-q794r\") pod \"payload-processing-external-model-68df898747-shn7h\" (UID: \"344725fe-7870-483c-b76b-74f4069f3386\") " pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h"
Nov 04 02:22:10.701222 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.701190    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-68df898747-shn7h"]
Nov 04 02:22:10.792069 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.792040    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/344725fe-7870-483c-b76b-74f4069f3386-plugins-config-volume\") pod \"payload-processing-external-model-68df898747-shn7h\" (UID: \"344725fe-7870-483c-b76b-74f4069f3386\") " pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h"
Nov 04 02:22:10.792228 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.792078    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/344725fe-7870-483c-b76b-74f4069f3386-provider-credentials\") pod \"payload-processing-external-model-68df898747-shn7h\" (UID: \"344725fe-7870-483c-b76b-74f4069f3386\") " pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h"
Nov 04 02:22:10.792228 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.792102    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q794r\" (UniqueName: \"kubernetes.io/projected/344725fe-7870-483c-b76b-74f4069f3386-kube-api-access-q794r\") pod \"payload-processing-external-model-68df898747-shn7h\" (UID: \"344725fe-7870-483c-b76b-74f4069f3386\") " pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h"
Nov 04 02:22:10.792228 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.792169    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/344725fe-7870-483c-b76b-74f4069f3386-routing-overlay-volume\") pod \"payload-processing-external-model-68df898747-shn7h\" (UID: \"344725fe-7870-483c-b76b-74f4069f3386\") " pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h"
Nov 04 02:22:10.792857 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.792831    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/344725fe-7870-483c-b76b-74f4069f3386-routing-overlay-volume\") pod \"payload-processing-external-model-68df898747-shn7h\" (UID: \"344725fe-7870-483c-b76b-74f4069f3386\") " pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h"
Nov 04 02:22:10.792960 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.792904    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/344725fe-7870-483c-b76b-74f4069f3386-plugins-config-volume\") pod \"payload-processing-external-model-68df898747-shn7h\" (UID: \"344725fe-7870-483c-b76b-74f4069f3386\") " pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h"
Nov 04 02:22:10.794600 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.794564    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/344725fe-7870-483c-b76b-74f4069f3386-provider-credentials\") pod \"payload-processing-external-model-68df898747-shn7h\" (UID: \"344725fe-7870-483c-b76b-74f4069f3386\") " pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h"
Nov 04 02:22:10.854018 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.853952    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q794r\" (UniqueName: \"kubernetes.io/projected/344725fe-7870-483c-b76b-74f4069f3386-kube-api-access-q794r\") pod \"payload-processing-external-model-68df898747-shn7h\" (UID: \"344725fe-7870-483c-b76b-74f4069f3386\") " pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h"
Nov 04 02:22:10.907928 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:10.907898    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h"
Nov 04 02:22:11.294425 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:11.294394    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-68df898747-shn7h"]
Nov 04 02:22:11.829304 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:11.828631    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" event={"ID":"344725fe-7870-483c-b76b-74f4069f3386","Type":"ContainerStarted","Data":"3bea87ed0e2f4d441f8d3981548b538f38304b8cd8e71371d62ad67be665770d"}
Nov 04 02:22:11.829304 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:11.828801    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" event={"ID":"344725fe-7870-483c-b76b-74f4069f3386","Type":"ContainerStarted","Data":"bf67c1809d7af4635a1e3d10dafca58dd8a7276f675169f036ac843fd3e7c641"}
Nov 04 02:22:11.829304 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:11.829205    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h"
Nov 04 02:22:11.947882 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:11.947832    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podStartSLOduration=1.947819168 podStartE2EDuration="1.947819168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:22:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:22:11.947068877 +0000 UTC m=+1595.173855570" watchObservedRunningTime="2026-11-04 02:22:11.947819168 +0000 UTC m=+1595.174605836"
Nov 04 02:22:15.761205 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:15.761171    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"]
Nov 04 02:22:22.371519 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.371485    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"]
Nov 04 02:22:22.375482 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.375450    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.474711 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.474666    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"]
Nov 04 02:22:22.544773 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.544740    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.544931 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.544808    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/441878e0-76c1-4e08-adb3-947bd44993d2-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.544931 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.544888    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/441878e0-76c1-4e08-adb3-947bd44993d2-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.544931 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.544924    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.545069 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.545047    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.545105 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.545086    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.545157 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.545146    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/441878e0-76c1-4e08-adb3-947bd44993d2-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.545194 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.545184    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqc95\" (UniqueName: \"kubernetes.io/projected/441878e0-76c1-4e08-adb3-947bd44993d2-kube-api-access-hqc95\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.545232 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.545202    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.667616 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.667533    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"]
Nov 04 02:22:22.668540 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:22.668505    4865 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[credential-socket istio-data istio-envoy istio-podinfo istio-token istiod-ca-cert kube-api-access-hqc95 workload-certs workload-socket], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf" podUID="441878e0-76c1-4e08-adb3-947bd44993d2"
Nov 04 02:22:22.671236 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.671181    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.671236 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.671222    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.671484 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.671260    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/441878e0-76c1-4e08-adb3-947bd44993d2-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.671484 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.671340    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqc95\" (UniqueName: \"kubernetes.io/projected/441878e0-76c1-4e08-adb3-947bd44993d2-kube-api-access-hqc95\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.671484 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.671366    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.671791 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.671765    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.671891 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.671822    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.671891 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.671877    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.672001 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.671910    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"]
Nov 04 02:22:22.672001 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.671984    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/441878e0-76c1-4e08-adb3-947bd44993d2-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.672103 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.672019    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/441878e0-76c1-4e08-adb3-947bd44993d2-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.672103 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.672057    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.672203 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.672153    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d" podUID="9049038a-28f6-48ac-9971-a60bfe207660" containerName="istio-proxy" containerID="cri-o://f5d9f2f3a47cd2542453e960282e29d2d46a385645d3276450f47f1b565e833a" gracePeriod=30
Nov 04 02:22:22.672254 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.672230    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.672890 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.672793    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.672890 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.672875    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/441878e0-76c1-4e08-adb3-947bd44993d2-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.674545 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.674522    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.675598 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.675573    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/441878e0-76c1-4e08-adb3-947bd44993d2-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.779518 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:22.779488    4865 projected.go:196] Error preparing data for projected volume istio-token for pod openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf: failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found
Nov 04 02:22:22.779716 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:22.779579    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/441878e0-76c1-4e08-adb3-947bd44993d2-istio-token podName:441878e0-76c1-4e08-adb3-947bd44993d2 nodeName:}" failed. No retries permitted until 2026-11-04 02:22:23.279557481 +0000 UTC m=+1606.506344127 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "istio-token" (UniqueName: "kubernetes.io/projected/441878e0-76c1-4e08-adb3-947bd44993d2-istio-token") pod "e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf" (UID: "441878e0-76c1-4e08-adb3-947bd44993d2") : failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found
Nov 04 02:22:22.779716 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:22.779583    4865 projected.go:196] Error preparing data for projected volume kube-api-access-hqc95 for pod openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf: failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found
Nov 04 02:22:22.779716 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:22.779667    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/441878e0-76c1-4e08-adb3-947bd44993d2-kube-api-access-hqc95 podName:441878e0-76c1-4e08-adb3-947bd44993d2 nodeName:}" failed. No retries permitted until 2026-11-04 02:22:23.279648237 +0000 UTC m=+1606.506434903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hqc95" (UniqueName: "kubernetes.io/projected/441878e0-76c1-4e08-adb3-947bd44993d2-kube-api-access-hqc95") pod "e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf" (UID: "441878e0-76c1-4e08-adb3-947bd44993d2") : failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found
Nov 04 02:22:22.836785 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.836747    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:22:22.950924 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.940062    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.967273 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.967236    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:22.976083 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.976048    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-istio-data\") pod \"441878e0-76c1-4e08-adb3-947bd44993d2\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") "
Nov 04 02:22:22.976294 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.976113    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/441878e0-76c1-4e08-adb3-947bd44993d2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/441878e0-76c1-4e08-adb3-947bd44993d2-istiod-ca-cert\") pod \"441878e0-76c1-4e08-adb3-947bd44993d2\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") "
Nov 04 02:22:22.976294 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.976229    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-istio-envoy\") pod \"441878e0-76c1-4e08-adb3-947bd44993d2\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") "
Nov 04 02:22:22.976416 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.976370    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/441878e0-76c1-4e08-adb3-947bd44993d2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/441878e0-76c1-4e08-adb3-947bd44993d2-istio-podinfo\") pod \"441878e0-76c1-4e08-adb3-947bd44993d2\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") "
Nov 04 02:22:22.976416 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.976403    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-workload-certs\") pod \"441878e0-76c1-4e08-adb3-947bd44993d2\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") "
Nov 04 02:22:22.976529 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.976441    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-credential-socket\") pod \"441878e0-76c1-4e08-adb3-947bd44993d2\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") "
Nov 04 02:22:22.976529 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.976457    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-workload-socket\") pod \"441878e0-76c1-4e08-adb3-947bd44993d2\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") "
Nov 04 02:22:22.976896 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.976874    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-workload-certs" pod "441878e0-76c1-4e08-adb3-947bd44993d2" (UID: "441878e0-76c1-4e08-adb3-947bd44993d2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:22.977367 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.977345    4865 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-workload-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:22.977650 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.977625    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-workload-socket" pod "441878e0-76c1-4e08-adb3-947bd44993d2" (UID: "441878e0-76c1-4e08-adb3-947bd44993d2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:22.977966 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.977711    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-istio-data" pod "441878e0-76c1-4e08-adb3-947bd44993d2" (UID: "441878e0-76c1-4e08-adb3-947bd44993d2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:22.977966 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.977726    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-credential-socket" pod "441878e0-76c1-4e08-adb3-947bd44993d2" (UID: "441878e0-76c1-4e08-adb3-947bd44993d2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:22.977966 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.977828    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/441878e0-76c1-4e08-adb3-947bd44993d2-istiod-ca-cert" pod "441878e0-76c1-4e08-adb3-947bd44993d2" (UID: "441878e0-76c1-4e08-adb3-947bd44993d2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:22:22.979831 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.979788    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-istio-envoy" pod "441878e0-76c1-4e08-adb3-947bd44993d2" (UID: "441878e0-76c1-4e08-adb3-947bd44993d2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:22.981403 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:22.981379    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/441878e0-76c1-4e08-adb3-947bd44993d2-istio-podinfo" pod "441878e0-76c1-4e08-adb3-947bd44993d2" (UID: "441878e0-76c1-4e08-adb3-947bd44993d2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:22:23.078618 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:23.078586    4865 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-istio-envoy\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:23.078618 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:23.078613    4865 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/441878e0-76c1-4e08-adb3-947bd44993d2-istio-podinfo\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:23.078618 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:23.078622    4865 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-credential-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:23.078848 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:23.078632    4865 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-workload-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:23.078848 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:23.078640    4865 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/441878e0-76c1-4e08-adb3-947bd44993d2-istio-data\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:23.078848 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:23.078649    4865 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/441878e0-76c1-4e08-adb3-947bd44993d2-istiod-ca-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:23.280662 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:23.280572    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/441878e0-76c1-4e08-adb3-947bd44993d2-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:23.280662 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:23.280634    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqc95\" (UniqueName: \"kubernetes.io/projected/441878e0-76c1-4e08-adb3-947bd44993d2-kube-api-access-hqc95\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf\" (UID: \"441878e0-76c1-4e08-adb3-947bd44993d2\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:23.334234 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:23.334210    4865 projected.go:196] Error preparing data for projected volume istio-token for pod openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf: failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found
Nov 04 02:22:23.334394 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:23.334271    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/441878e0-76c1-4e08-adb3-947bd44993d2-istio-token podName:441878e0-76c1-4e08-adb3-947bd44993d2 nodeName:}" failed. No retries permitted until 2026-11-04 02:22:24.334257017 +0000 UTC m=+1607.561043668 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "istio-token" (UniqueName: "kubernetes.io/projected/441878e0-76c1-4e08-adb3-947bd44993d2-istio-token") pod "e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf" (UID: "441878e0-76c1-4e08-adb3-947bd44993d2") : failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found
Nov 04 02:22:23.334452 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:23.334429    4865 projected.go:196] Error preparing data for projected volume kube-api-access-hqc95 for pod openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf: failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found
Nov 04 02:22:23.334506 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:23.334494    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/441878e0-76c1-4e08-adb3-947bd44993d2-kube-api-access-hqc95 podName:441878e0-76c1-4e08-adb3-947bd44993d2 nodeName:}" failed. No retries permitted until 2026-11-04 02:22:24.334480913 +0000 UTC m=+1607.561267565 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hqc95" (UniqueName: "kubernetes.io/projected/441878e0-76c1-4e08-adb3-947bd44993d2-kube-api-access-hqc95") pod "e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf" (UID: "441878e0-76c1-4e08-adb3-947bd44993d2") : failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found
Nov 04 02:22:23.945373 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:23.945343    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"
Nov 04 02:22:24.180454 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:24.180422    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"]
Nov 04 02:22:24.184591 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:24.184563    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf"]
Nov 04 02:22:24.292251 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:24.292187    4865 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/441878e0-76c1-4e08-adb3-947bd44993d2-istio-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:24.292251 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:24.292215    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqc95\" (UniqueName: \"kubernetes.io/projected/441878e0-76c1-4e08-adb3-947bd44993d2-kube-api-access-hqc95\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:24.896107 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:24.896072    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="441878e0-76c1-4e08-adb3-947bd44993d2" path="/var/lib/kubelet/pods/441878e0-76c1-4e08-adb3-947bd44993d2/volumes"
Nov 04 02:22:28.089358 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.088490    4865 generic.go:356] "Generic (PLEG): container finished" podID="9049038a-28f6-48ac-9971-a60bfe207660" containerID="f5d9f2f3a47cd2542453e960282e29d2d46a385645d3276450f47f1b565e833a" exitCode=0
Nov 04 02:22:28.089358 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.088589    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d" event={"ID":"9049038a-28f6-48ac-9971-a60bfe207660","Type":"ContainerDied","Data":"f5d9f2f3a47cd2542453e960282e29d2d46a385645d3276450f47f1b565e833a"}
Nov 04 02:22:28.089358 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.088650    4865 scope.go:122] "RemoveContainer" containerID="c1c4a2c059846ca23a5b2cc385a80a7d2ca0b39bc72d35628d0defff10705e1b"
Nov 04 02:22:28.446680 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:28.446586    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:22:28.475543 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.473298    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:22:28.536956 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.536922    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9049038a-28f6-48ac-9971-a60bfe207660-istio-token\" (UniqueName: \"kubernetes.io/projected/9049038a-28f6-48ac-9971-a60bfe207660-istio-token\") pod \"9049038a-28f6-48ac-9971-a60bfe207660\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") "
Nov 04 02:22:28.537153 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.536966    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-workload-socket\") pod \"9049038a-28f6-48ac-9971-a60bfe207660\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") "
Nov 04 02:22:28.537153 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.537016    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9049038a-28f6-48ac-9971-a60bfe207660-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9049038a-28f6-48ac-9971-a60bfe207660-istio-podinfo\") pod \"9049038a-28f6-48ac-9971-a60bfe207660\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") "
Nov 04 02:22:28.537153 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.537060    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-istio-envoy\") pod \"9049038a-28f6-48ac-9971-a60bfe207660\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") "
Nov 04 02:22:28.537153 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.537101    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9049038a-28f6-48ac-9971-a60bfe207660-kube-api-access-6kbl5\" (UniqueName: \"kubernetes.io/projected/9049038a-28f6-48ac-9971-a60bfe207660-kube-api-access-6kbl5\") pod \"9049038a-28f6-48ac-9971-a60bfe207660\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") "
Nov 04 02:22:28.537385 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.537162    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-workload-certs\") pod \"9049038a-28f6-48ac-9971-a60bfe207660\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") "
Nov 04 02:22:28.537385 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.537337    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9049038a-28f6-48ac-9971-a60bfe207660-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9049038a-28f6-48ac-9971-a60bfe207660-istiod-ca-cert\") pod \"9049038a-28f6-48ac-9971-a60bfe207660\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") "
Nov 04 02:22:28.537385 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.537367    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-credential-socket\") pod \"9049038a-28f6-48ac-9971-a60bfe207660\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") "
Nov 04 02:22:28.537540 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.537466    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-istio-data\") pod \"9049038a-28f6-48ac-9971-a60bfe207660\" (UID: \"9049038a-28f6-48ac-9971-a60bfe207660\") "
Nov 04 02:22:28.537768 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.537722    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-workload-certs" pod "9049038a-28f6-48ac-9971-a60bfe207660" (UID: "9049038a-28f6-48ac-9971-a60bfe207660"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:28.537913 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.537822    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-credential-socket" pod "9049038a-28f6-48ac-9971-a60bfe207660" (UID: "9049038a-28f6-48ac-9971-a60bfe207660"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:28.537988 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.537965    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-istio-data" pod "9049038a-28f6-48ac-9971-a60bfe207660" (UID: "9049038a-28f6-48ac-9971-a60bfe207660"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:28.538074 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.538050    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-workload-socket" pod "9049038a-28f6-48ac-9971-a60bfe207660" (UID: "9049038a-28f6-48ac-9971-a60bfe207660"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:28.538141 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.538087    4865 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-istio-data\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:28.538141 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.538109    4865 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-workload-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:28.538141 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.538122    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9049038a-28f6-48ac-9971-a60bfe207660-istiod-ca-cert" pod "9049038a-28f6-48ac-9971-a60bfe207660" (UID: "9049038a-28f6-48ac-9971-a60bfe207660"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:22:28.538141 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.538128    4865 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-credential-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:28.539511 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.539488    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9049038a-28f6-48ac-9971-a60bfe207660-istio-token" pod "9049038a-28f6-48ac-9971-a60bfe207660" (UID: "9049038a-28f6-48ac-9971-a60bfe207660"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:28.539786 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.539758    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9049038a-28f6-48ac-9971-a60bfe207660-istio-podinfo" pod "9049038a-28f6-48ac-9971-a60bfe207660" (UID: "9049038a-28f6-48ac-9971-a60bfe207660"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:22:28.540095 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.540077    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9049038a-28f6-48ac-9971-a60bfe207660-kube-api-access-6kbl5" pod "9049038a-28f6-48ac-9971-a60bfe207660" (UID: "9049038a-28f6-48ac-9971-a60bfe207660"). InnerVolumeSpecName "kube-api-access-6kbl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:28.540377 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.540357    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-istio-envoy" pod "9049038a-28f6-48ac-9971-a60bfe207660" (UID: "9049038a-28f6-48ac-9971-a60bfe207660"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:28.639261 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.639228    4865 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9049038a-28f6-48ac-9971-a60bfe207660-istiod-ca-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:28.639261 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.639259    4865 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9049038a-28f6-48ac-9971-a60bfe207660-istio-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:28.639261 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.639269    4865 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-workload-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:28.639522 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.639278    4865 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9049038a-28f6-48ac-9971-a60bfe207660-istio-podinfo\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:28.639522 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.639312    4865 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9049038a-28f6-48ac-9971-a60bfe207660-istio-envoy\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:28.639522 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:28.639321    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kbl5\" (UniqueName: \"kubernetes.io/projected/9049038a-28f6-48ac-9971-a60bfe207660-kube-api-access-6kbl5\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:29.095079 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:29.094996    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"
Nov 04 02:22:29.095079 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:29.095040    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d" event={"ID":"9049038a-28f6-48ac-9971-a60bfe207660","Type":"ContainerDied","Data":"44876a270feee6fb4eeeaba2891d9012702952e1610d2e9397a6051e52064f71"}
Nov 04 02:22:29.095587 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:29.095102    4865 scope.go:122] "RemoveContainer" containerID="f5d9f2f3a47cd2542453e960282e29d2d46a385645d3276450f47f1b565e833a"
Nov 04 02:22:29.263405 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:29.263373    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"]
Nov 04 02:22:29.313685 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:29.313651    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d"]
Nov 04 02:22:30.781443 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:30.781391    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj" podUID="ead7c7c3-0931-4bde-884b-69aa8dbd28e0" containerName="main" containerID="cri-o://c9a3a794712ae76f7082f0c4a1368f9abcd6d6d85b5eb74f0523ad92e8143956" gracePeriod=45
Nov 04 02:22:30.895113 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:30.895084    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9049038a-28f6-48ac-9971-a60bfe207660" path="/var/lib/kubelet/pods/9049038a-28f6-48ac-9971-a60bfe207660/volumes"
Nov 04 02:22:31.047084 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.047027    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:22:31.108234 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.108200    4865 generic.go:356] "Generic (PLEG): container finished" podID="ead7c7c3-0931-4bde-884b-69aa8dbd28e0" containerID="c9a3a794712ae76f7082f0c4a1368f9abcd6d6d85b5eb74f0523ad92e8143956" exitCode=0
Nov 04 02:22:31.108422 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.108240    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"
Nov 04 02:22:31.108422 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.108300    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj" event={"ID":"ead7c7c3-0931-4bde-884b-69aa8dbd28e0","Type":"ContainerDied","Data":"c9a3a794712ae76f7082f0c4a1368f9abcd6d6d85b5eb74f0523ad92e8143956"}
Nov 04 02:22:31.108422 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.108345    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj" event={"ID":"ead7c7c3-0931-4bde-884b-69aa8dbd28e0","Type":"ContainerDied","Data":"0be545a1acc87699722e1eaf9f16b090429f7696b6fc4c42e846b1a69c3061a7"}
Nov 04 02:22:31.108422 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.108366    4865 scope.go:122] "RemoveContainer" containerID="c9a3a794712ae76f7082f0c4a1368f9abcd6d6d85b5eb74f0523ad92e8143956"
Nov 04 02:22:31.116806 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.116787    4865 scope.go:122] "RemoveContainer" containerID="c9a3a794712ae76f7082f0c4a1368f9abcd6d6d85b5eb74f0523ad92e8143956"
Nov 04 02:22:31.117054 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:31.117031    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9a3a794712ae76f7082f0c4a1368f9abcd6d6d85b5eb74f0523ad92e8143956\": container with ID starting with c9a3a794712ae76f7082f0c4a1368f9abcd6d6d85b5eb74f0523ad92e8143956 not found: ID does not exist" containerID="c9a3a794712ae76f7082f0c4a1368f9abcd6d6d85b5eb74f0523ad92e8143956"
Nov 04 02:22:31.117098 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.117069    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9a3a794712ae76f7082f0c4a1368f9abcd6d6d85b5eb74f0523ad92e8143956"} err="failed to get container status \"c9a3a794712ae76f7082f0c4a1368f9abcd6d6d85b5eb74f0523ad92e8143956\": rpc error: code = NotFound desc = could not find container \"c9a3a794712ae76f7082f0c4a1368f9abcd6d6d85b5eb74f0523ad92e8143956\": container with ID starting with c9a3a794712ae76f7082f0c4a1368f9abcd6d6d85b5eb74f0523ad92e8143956 not found: ID does not exist"
Nov 04 02:22:31.183831 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.183802    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-home\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-home\") pod \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") "
Nov 04 02:22:31.183971 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.183857    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-model-cache\") pod \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") "
Nov 04 02:22:31.183971 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.183896    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-tls-certs\" (UniqueName: \"kubernetes.io/secret/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-tls-certs\") pod \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") "
Nov 04 02:22:31.183971 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.183957    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-dshm\") pod \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") "
Nov 04 02:22:31.184206 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.184050    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-kube-api-access-6nw45\" (UniqueName: \"kubernetes.io/projected/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-kube-api-access-6nw45\") pod \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") "
Nov 04 02:22:31.184206 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.184073    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-home" pod "ead7c7c3-0931-4bde-884b-69aa8dbd28e0" (UID: "ead7c7c3-0931-4bde-884b-69aa8dbd28e0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:31.184206 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.184131    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-tmp-dir\") pod \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\" (UID: \"ead7c7c3-0931-4bde-884b-69aa8dbd28e0\") "
Nov 04 02:22:31.184206 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.184156    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-model-cache" pod "ead7c7c3-0931-4bde-884b-69aa8dbd28e0" (UID: "ead7c7c3-0931-4bde-884b-69aa8dbd28e0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:31.184488 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.184463    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-tmp-dir" pod "ead7c7c3-0931-4bde-884b-69aa8dbd28e0" (UID: "ead7c7c3-0931-4bde-884b-69aa8dbd28e0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:31.184624 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.184552    4865 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-tmp-dir\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.184624 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.184568    4865 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-home\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.184624 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.184576    4865 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-model-cache\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.186246 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.186223    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-dshm" pod "ead7c7c3-0931-4bde-884b-69aa8dbd28e0" (UID: "ead7c7c3-0931-4bde-884b-69aa8dbd28e0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:31.186345 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.186233    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-tls-certs" pod "ead7c7c3-0931-4bde-884b-69aa8dbd28e0" (UID: "ead7c7c3-0931-4bde-884b-69aa8dbd28e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:22:31.186386 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.186351    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-kube-api-access-6nw45" pod "ead7c7c3-0931-4bde-884b-69aa8dbd28e0" (UID: "ead7c7c3-0931-4bde-884b-69aa8dbd28e0"). InnerVolumeSpecName "kube-api-access-6nw45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:31.285228 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.285194    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nw45\" (UniqueName: \"kubernetes.io/projected/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-kube-api-access-6nw45\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.285228 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.285224    4865 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-tls-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.285228 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.285233    4865 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ead7c7c3-0931-4bde-884b-69aa8dbd28e0-dshm\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.530393 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.530360    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"]
Nov 04 02:22:31.534703 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:31.534679    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/premium-w4-196e7e-kserve-5fb85f5b67-d2wkj"]
Nov 04 02:22:32.835253 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:32.835214    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:22:32.895601 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:32.895566    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ead7c7c3-0931-4bde-884b-69aa8dbd28e0" path="/var/lib/kubelet/pods/ead7c7c3-0931-4bde-884b-69aa8dbd28e0/volumes"
Nov 04 02:22:42.835570 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:42.835534    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:22:44.933602 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:44.933564    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:22:45.950958 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:45.950912    4865 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-ee7ade4978e6: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:22:45.950958 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:45.950952    4865 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:22:45.951517 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:45.951039    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials podName:742654db-eedb-4c8d-b61a-504ae2932fea nodeName:}" failed. No retries permitted until 2026-11-04 02:22:46.451017126 +0000 UTC m=+1629.677803773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials") pod "payload-processing-external-model-5f69dbdb4d-vvrwp" (UID: "742654db-eedb-4c8d-b61a-504ae2932fea") : secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:22:46.456300 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:46.456248    4865 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-ee7ade4978e6: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:22:46.456300 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:46.456275    4865 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:22:46.456499 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:46.456368    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials podName:742654db-eedb-4c8d-b61a-504ae2932fea nodeName:}" failed. No retries permitted until 2026-11-04 02:22:47.456354696 +0000 UTC m=+1630.683141342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials") pod "payload-processing-external-model-5f69dbdb4d-vvrwp" (UID: "742654db-eedb-4c8d-b61a-504ae2932fea") : secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:22:47.467880 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:47.467840    4865 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-ee7ade4978e6: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:22:47.467880 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:47.467872    4865 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:22:47.468340 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:47.467940    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials podName:742654db-eedb-4c8d-b61a-504ae2932fea nodeName:}" failed. No retries permitted until 2026-11-04 02:22:49.467925292 +0000 UTC m=+1632.694711951 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials") pod "payload-processing-external-model-5f69dbdb4d-vvrwp" (UID: "742654db-eedb-4c8d-b61a-504ae2932fea") : secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:22:49.494779 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:49.494745    4865 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-ee7ade4978e6: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:22:49.494779 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:49.494773    4865 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:22:49.495204 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:49.494838    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials podName:742654db-eedb-4c8d-b61a-504ae2932fea nodeName:}" failed. No retries permitted until 2026-11-04 02:22:53.494823564 +0000 UTC m=+1636.721610212 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials") pod "payload-processing-external-model-5f69dbdb4d-vvrwp" (UID: "742654db-eedb-4c8d-b61a-504ae2932fea") : secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:22:52.835531 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:52.835491    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:22:53.545228 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:53.545185    4865 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-ee7ade4978e6: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:22:53.545228 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:53.545216    4865 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:22:53.545478 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:53.545322    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials podName:742654db-eedb-4c8d-b61a-504ae2932fea nodeName:}" failed. No retries permitted until 2026-11-04 02:23:01.545261296 +0000 UTC m=+1644.772047956 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials") pod "payload-processing-external-model-5f69dbdb4d-vvrwp" (UID: "742654db-eedb-4c8d-b61a-504ae2932fea") : secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:22:57.032690 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:57.032647    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"]
Nov 04 02:22:57.033181 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:57.032904    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk" podUID="f207e785-019b-4488-8ef4-8a0f56f8134d" containerName="maas-api" containerID="cri-o://8f4d31e8f43522fba2f7d5b78e4765210198b5624df103e0ceb0ee19c5627252" gracePeriod=30
Nov 04 02:22:57.295634 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:57.295569    4865 generic.go:356] "Generic (PLEG): container finished" podID="f207e785-019b-4488-8ef4-8a0f56f8134d" containerID="8f4d31e8f43522fba2f7d5b78e4765210198b5624df103e0ceb0ee19c5627252" exitCode=0
Nov 04 02:22:57.295634 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:57.295615    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk" event={"ID":"f207e785-019b-4488-8ef4-8a0f56f8134d","Type":"ContainerDied","Data":"8f4d31e8f43522fba2f7d5b78e4765210198b5624df103e0ceb0ee19c5627252"}
Nov 04 02:22:57.317924 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:57.317898    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:22:57.391652 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:57.391624    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f207e785-019b-4488-8ef4-8a0f56f8134d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f207e785-019b-4488-8ef4-8a0f56f8134d-metrics-certs\") pod \"f207e785-019b-4488-8ef4-8a0f56f8134d\" (UID: \"f207e785-019b-4488-8ef4-8a0f56f8134d\") "
Nov 04 02:22:57.391830 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:57.391660    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f207e785-019b-4488-8ef4-8a0f56f8134d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f207e785-019b-4488-8ef4-8a0f56f8134d-openshift-service-ca\") pod \"f207e785-019b-4488-8ef4-8a0f56f8134d\" (UID: \"f207e785-019b-4488-8ef4-8a0f56f8134d\") "
Nov 04 02:22:57.391830 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:57.391791    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f207e785-019b-4488-8ef4-8a0f56f8134d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f207e785-019b-4488-8ef4-8a0f56f8134d-maas-api-tls\") pod \"f207e785-019b-4488-8ef4-8a0f56f8134d\" (UID: \"f207e785-019b-4488-8ef4-8a0f56f8134d\") "
Nov 04 02:22:57.391949 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:57.391859    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f207e785-019b-4488-8ef4-8a0f56f8134d-kube-api-access-7pv7w\" (UniqueName: \"kubernetes.io/projected/f207e785-019b-4488-8ef4-8a0f56f8134d-kube-api-access-7pv7w\") pod \"f207e785-019b-4488-8ef4-8a0f56f8134d\" (UID: \"f207e785-019b-4488-8ef4-8a0f56f8134d\") "
Nov 04 02:22:57.392095 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:57.392064    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f207e785-019b-4488-8ef4-8a0f56f8134d-openshift-service-ca" pod "f207e785-019b-4488-8ef4-8a0f56f8134d" (UID: "f207e785-019b-4488-8ef4-8a0f56f8134d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:22:57.392324 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:57.392306    4865 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f207e785-019b-4488-8ef4-8a0f56f8134d-openshift-service-ca\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:57.393942 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:57.393912    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f207e785-019b-4488-8ef4-8a0f56f8134d-maas-api-tls" pod "f207e785-019b-4488-8ef4-8a0f56f8134d" (UID: "f207e785-019b-4488-8ef4-8a0f56f8134d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:22:57.393942 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:57.393934    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f207e785-019b-4488-8ef4-8a0f56f8134d-metrics-certs" pod "f207e785-019b-4488-8ef4-8a0f56f8134d" (UID: "f207e785-019b-4488-8ef4-8a0f56f8134d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:22:57.394110 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:57.393939    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f207e785-019b-4488-8ef4-8a0f56f8134d-kube-api-access-7pv7w" pod "f207e785-019b-4488-8ef4-8a0f56f8134d" (UID: "f207e785-019b-4488-8ef4-8a0f56f8134d"). InnerVolumeSpecName "kube-api-access-7pv7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:57.493975 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:57.493933    4865 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f207e785-019b-4488-8ef4-8a0f56f8134d-metrics-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:57.493975 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:57.493962    4865 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f207e785-019b-4488-8ef4-8a0f56f8134d-maas-api-tls\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:57.493975 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:57.493973    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pv7w\" (UniqueName: \"kubernetes.io/projected/f207e785-019b-4488-8ef4-8a0f56f8134d-kube-api-access-7pv7w\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:58.302133 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:58.302095    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"
Nov 04 02:22:58.302641 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:58.302129    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk" event={"ID":"f207e785-019b-4488-8ef4-8a0f56f8134d","Type":"ContainerDied","Data":"b95706c050ce02791c5a498f6b0284685709b93989cd00e3e533f0cf1a90aaea"}
Nov 04 02:22:58.302641 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:58.302191    4865 scope.go:122] "RemoveContainer" containerID="8f4d31e8f43522fba2f7d5b78e4765210198b5624df103e0ceb0ee19c5627252"
Nov 04 02:22:58.404493 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:22:58.404455    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:22:58.469578 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:58.469541    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"]
Nov 04 02:22:58.520322 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:58.520272    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-196e7e-79cc8df857-chxrk"]
Nov 04 02:22:58.896434 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:22:58.896403    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f207e785-019b-4488-8ef4-8a0f56f8134d" path="/var/lib/kubelet/pods/f207e785-019b-4488-8ef4-8a0f56f8134d/volumes"
Nov 04 02:23:01.645636 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:23:01.645589    4865 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-ee7ade4978e6: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:23:01.645636 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:23:01.645625    4865 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:23:01.646180 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:23:01.645690    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials podName:742654db-eedb-4c8d-b61a-504ae2932fea nodeName:}" failed. No retries permitted until 2026-11-04 02:23:17.645672304 +0000 UTC m=+1660.872458967 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials") pod "payload-processing-external-model-5f69dbdb4d-vvrwp" (UID: "742654db-eedb-4c8d-b61a-504ae2932fea") : secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:23:02.835604 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:02.835565    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:23:07.507117 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.507081    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"]
Nov 04 02:23:07.507711 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.507692    4865 cpu_manager.go:414] "containerMap: removing container" podUID="9049038a-28f6-48ac-9971-a60bfe207660" containerName="istio-proxy"
Nov 04 02:23:07.507711 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.507712    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9049038a-28f6-48ac-9971-a60bfe207660" containerName="istio-proxy"
Nov 04 02:23:07.507839 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.507722    4865 cpu_manager.go:414] "containerMap: removing container" podUID="ead7c7c3-0931-4bde-884b-69aa8dbd28e0" containerName="main"
Nov 04 02:23:07.507839 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.507727    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ead7c7c3-0931-4bde-884b-69aa8dbd28e0" containerName="main"
Nov 04 02:23:07.507839 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.507746    4865 cpu_manager.go:414] "containerMap: removing container" podUID="f207e785-019b-4488-8ef4-8a0f56f8134d" containerName="maas-api"
Nov 04 02:23:07.507839 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.507752    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f207e785-019b-4488-8ef4-8a0f56f8134d" containerName="maas-api"
Nov 04 02:23:07.507839 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.507763    4865 cpu_manager.go:414] "containerMap: removing container" podUID="9049038a-28f6-48ac-9971-a60bfe207660" containerName="istio-proxy"
Nov 04 02:23:07.507839 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.507768    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9049038a-28f6-48ac-9971-a60bfe207660" containerName="istio-proxy"
Nov 04 02:23:07.508112 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.507871    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="ead7c7c3-0931-4bde-884b-69aa8dbd28e0" containerName="main"
Nov 04 02:23:07.508112 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.507883    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="9049038a-28f6-48ac-9971-a60bfe207660" containerName="istio-proxy"
Nov 04 02:23:07.508112 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.507892    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="f207e785-019b-4488-8ef4-8a0f56f8134d" containerName="maas-api"
Nov 04 02:23:07.512096 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.512072    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.609550 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.609511    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-token\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.609726 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.609610    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-credential-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.609726 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.609689    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istiod-ca-cert\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.609726 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.609719    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-data\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.609914 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.609831    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-podinfo\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.609914 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.609875    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htlzv\" (UniqueName: \"kubernetes.io/projected/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-kube-api-access-htlzv\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.610027 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.609935    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-workload-certs\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.610027 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.610002    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-workload-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.610124 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.610087    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-envoy\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.618642 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.618617    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"]
Nov 04 02:23:07.710810 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.710770    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-podinfo\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.711004 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.710821    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htlzv\" (UniqueName: \"kubernetes.io/projected/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-kube-api-access-htlzv\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.711004 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.710847    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-workload-certs\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.711004 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.710871    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-workload-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.711004 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.710942    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-envoy\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.711230 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.711043    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-token\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.711230 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.711073    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-credential-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.711230 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.711124    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istiod-ca-cert\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.711230 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.711169    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-data\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.711467 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.711352    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-workload-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.711467 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.711385    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-workload-certs\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.711467 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.711442    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-credential-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.711624 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.711558    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-data\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.712051 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.712027    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istiod-ca-cert\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.713303 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.713263    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-envoy\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.713424 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.713408    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-podinfo\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.772718 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.772646    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-token\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.772839 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.772761    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htlzv\" (UniqueName: \"kubernetes.io/projected/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-kube-api-access-htlzv\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:07.826988 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:07.826938    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:08.257486 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:08.257454    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"]
Nov 04 02:23:08.258614 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:23:08.258592    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc22a4702_2ab7_4eb4_bb1f_64ee45937a41.slice/crio-1ea144b9bb9a53f624f505238c2651d906d9d79f36a24b61ad1899b6de478a69 WatchSource:0}: Error finding container 1ea144b9bb9a53f624f505238c2651d906d9d79f36a24b61ad1899b6de478a69: Status 404 returned error can't find the container with id 1ea144b9bb9a53f624f505238c2651d906d9d79f36a24b61ad1899b6de478a69
Nov 04 02:23:08.260796 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:08.260766    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:23:08.260878 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:08.260831    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:23:08.260878 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:08.260864    4865 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 02:23:08.369326 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:08.369265    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x" event={"ID":"c22a4702-2ab7-4eb4-bb1f-64ee45937a41","Type":"ContainerStarted","Data":"061c3cff643d5398360bc7c3cccc4479d0cfe44f041b66277ab61b0b0763f443"}
Nov 04 02:23:08.369489 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:08.369348    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x" event={"ID":"c22a4702-2ab7-4eb4-bb1f-64ee45937a41","Type":"ContainerStarted","Data":"1ea144b9bb9a53f624f505238c2651d906d9d79f36a24b61ad1899b6de478a69"}
Nov 04 02:23:08.498456 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:08.498382    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x" podStartSLOduration=1.498362786 podStartE2EDuration="1.498362786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:23:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:23:08.495631693 +0000 UTC m=+1651.722418359" watchObservedRunningTime="2026-11-04 02:23:08.498362786 +0000 UTC m=+1651.725149455"
Nov 04 02:23:08.827695 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:08.827650    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:09.832385 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:09.832344    4865 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:10.380317 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:10.380249    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:10.381266 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:10.381243    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:12.835863 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:12.835822    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:23:17.169129 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:17.169033    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"]
Nov 04 02:23:17.169629 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:17.169211    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv" podUID="42661d7a-b751-4a93-b66c-25c7d0275abf" containerName="istio-proxy" containerID="cri-o://3857d37b28639e897ea46867f520dd1eafc8efa0c88a57340ce4afd237f5faeb" gracePeriod=30
Nov 04 02:23:17.171319 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:17.171293    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"]
Nov 04 02:23:17.171536 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:17.171513    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x" podUID="c22a4702-2ab7-4eb4-bb1f-64ee45937a41" containerName="istio-proxy" containerID="cri-o://061c3cff643d5398360bc7c3cccc4479d0cfe44f041b66277ab61b0b0763f443" gracePeriod=30
Nov 04 02:23:17.527617 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:17.527585    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:23:17.746979 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:23:17.746925    4865 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-ee7ade4978e6: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:23:17.746979 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:23:17.746979    4865 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:23:17.747207 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:23:17.747044    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials podName:742654db-eedb-4c8d-b61a-504ae2932fea nodeName:}" failed. No retries permitted until 2026-11-04 02:23:49.747029589 +0000 UTC m=+1692.973816249 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials") pod "payload-processing-external-model-5f69dbdb4d-vvrwp" (UID: "742654db-eedb-4c8d-b61a-504ae2932fea") : secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:23:22.438981 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.438911    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:22.443377 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.443344    4865 generic.go:356] "Generic (PLEG): container finished" podID="c22a4702-2ab7-4eb4-bb1f-64ee45937a41" containerID="061c3cff643d5398360bc7c3cccc4479d0cfe44f041b66277ab61b0b0763f443" exitCode=0
Nov 04 02:23:22.443523 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.443387    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"
Nov 04 02:23:22.443523 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.443419    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x" event={"ID":"c22a4702-2ab7-4eb4-bb1f-64ee45937a41","Type":"ContainerDied","Data":"061c3cff643d5398360bc7c3cccc4479d0cfe44f041b66277ab61b0b0763f443"}
Nov 04 02:23:22.443523 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.443487    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x" event={"ID":"c22a4702-2ab7-4eb4-bb1f-64ee45937a41","Type":"ContainerDied","Data":"1ea144b9bb9a53f624f505238c2651d906d9d79f36a24b61ad1899b6de478a69"}
Nov 04 02:23:22.443523 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.443518    4865 scope.go:122] "RemoveContainer" containerID="061c3cff643d5398360bc7c3cccc4479d0cfe44f041b66277ab61b0b0763f443"
Nov 04 02:23:22.445722 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.445701    4865 generic.go:356] "Generic (PLEG): container finished" podID="42661d7a-b751-4a93-b66c-25c7d0275abf" containerID="3857d37b28639e897ea46867f520dd1eafc8efa0c88a57340ce4afd237f5faeb" exitCode=0
Nov 04 02:23:22.445825 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.445740    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv" event={"ID":"42661d7a-b751-4a93-b66c-25c7d0275abf","Type":"ContainerDied","Data":"3857d37b28639e897ea46867f520dd1eafc8efa0c88a57340ce4afd237f5faeb"}
Nov 04 02:23:22.455368 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.453214    4865 scope.go:122] "RemoveContainer" containerID="061c3cff643d5398360bc7c3cccc4479d0cfe44f041b66277ab61b0b0763f443"
Nov 04 02:23:22.455368 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:23:22.455346    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"061c3cff643d5398360bc7c3cccc4479d0cfe44f041b66277ab61b0b0763f443\": container with ID starting with 061c3cff643d5398360bc7c3cccc4479d0cfe44f041b66277ab61b0b0763f443 not found: ID does not exist" containerID="061c3cff643d5398360bc7c3cccc4479d0cfe44f041b66277ab61b0b0763f443"
Nov 04 02:23:22.455491 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.455382    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061c3cff643d5398360bc7c3cccc4479d0cfe44f041b66277ab61b0b0763f443"} err="failed to get container status \"061c3cff643d5398360bc7c3cccc4479d0cfe44f041b66277ab61b0b0763f443\": rpc error: code = NotFound desc = could not find container \"061c3cff643d5398360bc7c3cccc4479d0cfe44f041b66277ab61b0b0763f443\": container with ID starting with 061c3cff643d5398360bc7c3cccc4479d0cfe44f041b66277ab61b0b0763f443 not found: ID does not exist"
Nov 04 02:23:22.455491 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.455407    4865 scope.go:122] "RemoveContainer" containerID="0e21b016d78ba025c5f70b5861e4eb0e917053b9a02aa845a43a4467c751d740"
Nov 04 02:23:22.487210 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.487183    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:23:22.498410 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.498388    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-envoy\") pod \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") "
Nov 04 02:23:22.498528 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.498511    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-podinfo\") pod \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") "
Nov 04 02:23:22.498573 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.498552    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-workload-certs\") pod \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") "
Nov 04 02:23:22.498616 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.498598    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-kube-api-access-htlzv\" (UniqueName: \"kubernetes.io/projected/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-kube-api-access-htlzv\") pod \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") "
Nov 04 02:23:22.498660 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.498639    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-workload-socket\") pod \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") "
Nov 04 02:23:22.498713 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.498688    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-data\") pod \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") "
Nov 04 02:23:22.498769 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.498726    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-credential-socket\") pod \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") "
Nov 04 02:23:22.498981 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.498924    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-workload-certs" pod "c22a4702-2ab7-4eb4-bb1f-64ee45937a41" (UID: "c22a4702-2ab7-4eb4-bb1f-64ee45937a41"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:22.498981 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.498934    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-workload-socket" pod "c22a4702-2ab7-4eb4-bb1f-64ee45937a41" (UID: "c22a4702-2ab7-4eb4-bb1f-64ee45937a41"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:22.499161 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.498953    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istiod-ca-cert\") pod \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") "
Nov 04 02:23:22.499161 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.499027    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-data" pod "c22a4702-2ab7-4eb4-bb1f-64ee45937a41" (UID: "c22a4702-2ab7-4eb4-bb1f-64ee45937a41"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:22.499161 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.499081    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-token\" (UniqueName: \"kubernetes.io/projected/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-token\") pod \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\" (UID: \"c22a4702-2ab7-4eb4-bb1f-64ee45937a41\") "
Nov 04 02:23:22.499161 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.499092    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-credential-socket" pod "c22a4702-2ab7-4eb4-bb1f-64ee45937a41" (UID: "c22a4702-2ab7-4eb4-bb1f-64ee45937a41"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:22.499381 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.499276    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istiod-ca-cert" pod "c22a4702-2ab7-4eb4-bb1f-64ee45937a41" (UID: "c22a4702-2ab7-4eb4-bb1f-64ee45937a41"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:23:22.499549 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.499529    4865 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-workload-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.499617 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.499553    4865 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-workload-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.499617 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.499567    4865 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-data\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.499617 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.499582    4865 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-credential-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.499617 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.499596    4865 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istiod-ca-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.500710 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.500687    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-kube-api-access-htlzv" pod "c22a4702-2ab7-4eb4-bb1f-64ee45937a41" (UID: "c22a4702-2ab7-4eb4-bb1f-64ee45937a41"). InnerVolumeSpecName "kube-api-access-htlzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:23:22.500793 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.500689    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-podinfo" pod "c22a4702-2ab7-4eb4-bb1f-64ee45937a41" (UID: "c22a4702-2ab7-4eb4-bb1f-64ee45937a41"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:23:22.500921 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.500896    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-envoy" pod "c22a4702-2ab7-4eb4-bb1f-64ee45937a41" (UID: "c22a4702-2ab7-4eb4-bb1f-64ee45937a41"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:22.501202 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.501189    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-token" pod "c22a4702-2ab7-4eb4-bb1f-64ee45937a41" (UID: "c22a4702-2ab7-4eb4-bb1f-64ee45937a41"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:23:22.600327 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.600271    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42661d7a-b751-4a93-b66c-25c7d0275abf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42661d7a-b751-4a93-b66c-25c7d0275abf-istiod-ca-cert\") pod \"42661d7a-b751-4a93-b66c-25c7d0275abf\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") "
Nov 04 02:23:22.600519 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.600334    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-workload-socket\") pod \"42661d7a-b751-4a93-b66c-25c7d0275abf\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") "
Nov 04 02:23:22.600519 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.600362    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-workload-certs\") pod \"42661d7a-b751-4a93-b66c-25c7d0275abf\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") "
Nov 04 02:23:22.600519 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.600433    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-credential-socket\") pod \"42661d7a-b751-4a93-b66c-25c7d0275abf\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") "
Nov 04 02:23:22.600519 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.600475    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-envoy\") pod \"42661d7a-b751-4a93-b66c-25c7d0275abf\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") "
Nov 04 02:23:22.600519 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.600511    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42661d7a-b751-4a93-b66c-25c7d0275abf-kube-api-access-556cf\" (UniqueName: \"kubernetes.io/projected/42661d7a-b751-4a93-b66c-25c7d0275abf-kube-api-access-556cf\") pod \"42661d7a-b751-4a93-b66c-25c7d0275abf\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") "
Nov 04 02:23:22.600803 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.600618    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-token\" (UniqueName: \"kubernetes.io/projected/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-token\") pod \"42661d7a-b751-4a93-b66c-25c7d0275abf\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") "
Nov 04 02:23:22.600803 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.600649    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-podinfo\") pod \"42661d7a-b751-4a93-b66c-25c7d0275abf\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") "
Nov 04 02:23:22.600803 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.600685    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-data\") pod \"42661d7a-b751-4a93-b66c-25c7d0275abf\" (UID: \"42661d7a-b751-4a93-b66c-25c7d0275abf\") "
Nov 04 02:23:22.601076 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.600678    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-workload-socket" pod "42661d7a-b751-4a93-b66c-25c7d0275abf" (UID: "42661d7a-b751-4a93-b66c-25c7d0275abf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:22.601076 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.600729    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-credential-socket" pod "42661d7a-b751-4a93-b66c-25c7d0275abf" (UID: "42661d7a-b751-4a93-b66c-25c7d0275abf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:22.601076 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.600740    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-workload-certs" pod "42661d7a-b751-4a93-b66c-25c7d0275abf" (UID: "42661d7a-b751-4a93-b66c-25c7d0275abf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:22.601076 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.600968    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-data" pod "42661d7a-b751-4a93-b66c-25c7d0275abf" (UID: "42661d7a-b751-4a93-b66c-25c7d0275abf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:22.601076 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.601023    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42661d7a-b751-4a93-b66c-25c7d0275abf-istiod-ca-cert" pod "42661d7a-b751-4a93-b66c-25c7d0275abf" (UID: "42661d7a-b751-4a93-b66c-25c7d0275abf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:23:22.601460 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.601225    4865 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.601460 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.601245    4865 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-envoy\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.601460 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.601260    4865 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-data\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.601460 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.601276    4865 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-istio-podinfo\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.601460 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.601305    4865 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42661d7a-b751-4a93-b66c-25c7d0275abf-istiod-ca-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.601460 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.601319    4865 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-workload-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.601460 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.601332    4865 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-workload-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.601460 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.601346    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htlzv\" (UniqueName: \"kubernetes.io/projected/c22a4702-2ab7-4eb4-bb1f-64ee45937a41-kube-api-access-htlzv\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.601460 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.601359    4865 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-credential-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.602848 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.602817    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42661d7a-b751-4a93-b66c-25c7d0275abf-kube-api-access-556cf" pod "42661d7a-b751-4a93-b66c-25c7d0275abf" (UID: "42661d7a-b751-4a93-b66c-25c7d0275abf"). InnerVolumeSpecName "kube-api-access-556cf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:23:22.602942 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.602899    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-podinfo" pod "42661d7a-b751-4a93-b66c-25c7d0275abf" (UID: "42661d7a-b751-4a93-b66c-25c7d0275abf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:23:22.603151 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.603126    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-envoy" pod "42661d7a-b751-4a93-b66c-25c7d0275abf" (UID: "42661d7a-b751-4a93-b66c-25c7d0275abf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:22.603151 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.603143    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-token" pod "42661d7a-b751-4a93-b66c-25c7d0275abf" (UID: "42661d7a-b751-4a93-b66c-25c7d0275abf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:23:22.702852 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.702816    4865 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-envoy\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.702852 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.702847    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-556cf\" (UniqueName: \"kubernetes.io/projected/42661d7a-b751-4a93-b66c-25c7d0275abf-kube-api-access-556cf\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.702852 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.702857    4865 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.703094 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.702865    4865 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42661d7a-b751-4a93-b66c-25c7d0275abf-istio-podinfo\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.835634 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.835597    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:23:22.904223 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.904193    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"]
Nov 04 02:23:22.909539 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:22.909517    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x"]
Nov 04 02:23:23.453067 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:23.453031    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv" event={"ID":"42661d7a-b751-4a93-b66c-25c7d0275abf","Type":"ContainerDied","Data":"5d54084dc1cbd10261db3d0d824ab39103f6f143ead566fd4e6d3cadbc5a5e21"}
Nov 04 02:23:23.453067 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:23.453060    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"
Nov 04 02:23:23.453636 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:23.453090    4865 scope.go:122] "RemoveContainer" containerID="3857d37b28639e897ea46867f520dd1eafc8efa0c88a57340ce4afd237f5faeb"
Nov 04 02:23:23.622786 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:23.622752    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"]
Nov 04 02:23:23.673308 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:23.673262    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv"]
Nov 04 02:23:24.896154 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:24.896119    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42661d7a-b751-4a93-b66c-25c7d0275abf" path="/var/lib/kubelet/pods/42661d7a-b751-4a93-b66c-25c7d0275abf/volumes"
Nov 04 02:23:24.896621 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:24.896606    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c22a4702-2ab7-4eb4-bb1f-64ee45937a41" path="/var/lib/kubelet/pods/c22a4702-2ab7-4eb4-bb1f-64ee45937a41/volumes"
Nov 04 02:23:28.403791 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:23:28.403756    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:23:32.836061 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:32.836018    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:23:37.890359 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:37.890324    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:23:47.891098 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:47.891055    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:23:49.802682 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:23:49.802651    4865 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-ee7ade4978e6: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:23:49.802682 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:23:49.802676    4865 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:23:49.803107 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:23:49.802740    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials podName:742654db-eedb-4c8d-b61a-504ae2932fea nodeName:}" failed. No retries permitted until 2026-11-04 02:24:53.802723373 +0000 UTC m=+1757.029510036 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials") pod "payload-processing-external-model-5f69dbdb4d-vvrwp" (UID: "742654db-eedb-4c8d-b61a-504ae2932fea") : secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:23:57.890243 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:23:57.890201    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:23:58.403995 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:23:58.403958    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:24:07.893597 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:24:07.893540    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:24:17.889950 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:24:17.889906    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:24:27.890089 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:24:27.890041    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:24:28.404552 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:24:28.404516    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:24:37.890031 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:24:37.889987    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:24:44.890661 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:24:44.890571    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:24:53.804096 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:24:53.804064    4865 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-ee7ade4978e6: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:24:53.804096 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:24:53.804087    4865 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:24:53.804535 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:24:53.804139    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials podName:742654db-eedb-4c8d-b61a-504ae2932fea nodeName:}" failed. No retries permitted until 2026-11-04 02:26:55.804124289 +0000 UTC m=+1879.030910936 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials") pod "payload-processing-external-model-5f69dbdb4d-vvrwp" (UID: "742654db-eedb-4c8d-b61a-504ae2932fea") : secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:24:54.890963 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:24:54.890924    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:24:58.403318 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:24:58.403252    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:25:04.890759 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:25:04.890718    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:25:14.890898 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:25:14.890850    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:25:24.891308 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:25:24.891243    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:25:28.404015 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:25:28.403973    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:25:34.890701 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:25:34.890653    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:25:37.081323 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:25:37.081278    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/2.log"
Nov 04 02:25:37.089927 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:25:37.089898    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/2.log"
Nov 04 02:25:44.890937 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:25:44.890890    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:25:54.890882 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:25:54.890838    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:25:58.404842 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:25:58.404802    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:26:02.889651 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:26:02.889611    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:26:12.890013 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:26:12.889969    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:26:22.890796 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:26:22.890706    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:26:28.403993 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:26:28.403956    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:26:32.890214 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:26:32.890167    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:26:42.889802 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:26:42.889764    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:26:52.890335 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:26:52.890268    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:26:55.866430 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:26:55.866389    4865 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-ee7ade4978e6: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:26:55.866430 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:26:55.866415    4865 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp: secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:26:55.866969 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:26:55.866479    4865 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials podName:742654db-eedb-4c8d-b61a-504ae2932fea nodeName:}" failed. No retries permitted until 2026-11-04 02:28:57.866461228 +0000 UTC m=+2001.093247900 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials") pod "payload-processing-external-model-5f69dbdb4d-vvrwp" (UID: "742654db-eedb-4c8d-b61a-504ae2932fea") : secret "agc-cred-ee7ade4978e6" not found
Nov 04 02:26:58.403974 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:26:58.403934    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:27:02.889661 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:02.889620    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:27:12.889980 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:12.889924    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:27:22.889733 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:22.889682    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:27:28.404026 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:27:28.403987    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:27:31.889659 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:31.889627    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:27:38.393164 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:38.393126    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"]
Nov 04 02:27:38.443598 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:38.443564    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-68df898747-shn7h"]
Nov 04 02:27:38.445026 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:38.444997    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:27:40.707652 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:40.707616    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:27:43.396302 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:43.395355    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp" podUID="742654db-eedb-4c8d-b61a-504ae2932fea" containerName="payload-processing" containerID="cri-o://3b2bfc056ff4b0e23ad26ca200f6398d5cd15fc7398081328fdb6e4e19099887" gracePeriod=25
Nov 04 02:27:43.450462 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:43.450377    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing" containerID="cri-o://3bea87ed0e2f4d441f8d3981548b538f38304b8cd8e71371d62ad67be665770d" gracePeriod=25
Nov 04 02:27:43.856307 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:43.840814    4865 generic.go:356] "Generic (PLEG): container finished" podID="742654db-eedb-4c8d-b61a-504ae2932fea" containerID="3b2bfc056ff4b0e23ad26ca200f6398d5cd15fc7398081328fdb6e4e19099887" exitCode=0
Nov 04 02:27:43.856307 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:43.840944    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp" event={"ID":"742654db-eedb-4c8d-b61a-504ae2932fea","Type":"ContainerDied","Data":"3b2bfc056ff4b0e23ad26ca200f6398d5cd15fc7398081328fdb6e4e19099887"}
Nov 04 02:27:43.856307 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:43.843407    4865 generic.go:356] "Generic (PLEG): container finished" podID="344725fe-7870-483c-b76b-74f4069f3386" containerID="3bea87ed0e2f4d441f8d3981548b538f38304b8cd8e71371d62ad67be665770d" exitCode=0
Nov 04 02:27:43.856307 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:43.843549    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" event={"ID":"344725fe-7870-483c-b76b-74f4069f3386","Type":"ContainerDied","Data":"3bea87ed0e2f4d441f8d3981548b538f38304b8cd8e71371d62ad67be665770d"}
Nov 04 02:27:44.527107 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.526769    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:27:44.569777 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.569737    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-kube-api-access-tq7rg\" (UniqueName: \"kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-kube-api-access-tq7rg\") pod \"742654db-eedb-4c8d-b61a-504ae2932fea\" (UID: \"742654db-eedb-4c8d-b61a-504ae2932fea\") "
Nov 04 02:27:44.573692 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.573662    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/742654db-eedb-4c8d-b61a-504ae2932fea-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/742654db-eedb-4c8d-b61a-504ae2932fea-plugins-config-volume\") pod \"742654db-eedb-4c8d-b61a-504ae2932fea\" (UID: \"742654db-eedb-4c8d-b61a-504ae2932fea\") "
Nov 04 02:27:44.573999 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.573980    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/742654db-eedb-4c8d-b61a-504ae2932fea-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/742654db-eedb-4c8d-b61a-504ae2932fea-routing-overlay-volume\") pod \"742654db-eedb-4c8d-b61a-504ae2932fea\" (UID: \"742654db-eedb-4c8d-b61a-504ae2932fea\") "
Nov 04 02:27:44.574120 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.574106    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials\" (UniqueName: \"kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials\") pod \"742654db-eedb-4c8d-b61a-504ae2932fea\" (UID: \"742654db-eedb-4c8d-b61a-504ae2932fea\") "
Nov 04 02:27:44.575602 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.575579    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/742654db-eedb-4c8d-b61a-504ae2932fea-routing-overlay-volume" pod "742654db-eedb-4c8d-b61a-504ae2932fea" (UID: "742654db-eedb-4c8d-b61a-504ae2932fea"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:27:44.576103 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.576086    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/742654db-eedb-4c8d-b61a-504ae2932fea-plugins-config-volume" pod "742654db-eedb-4c8d-b61a-504ae2932fea" (UID: "742654db-eedb-4c8d-b61a-504ae2932fea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:27:44.583649 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.583597    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-kube-api-access-tq7rg" pod "742654db-eedb-4c8d-b61a-504ae2932fea" (UID: "742654db-eedb-4c8d-b61a-504ae2932fea"). InnerVolumeSpecName "kube-api-access-tq7rg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:27:44.584048 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.584009    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials" pod "742654db-eedb-4c8d-b61a-504ae2932fea" (UID: "742654db-eedb-4c8d-b61a-504ae2932fea"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:27:44.618042 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.618002    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h"
Nov 04 02:27:44.675227 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.675138    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/344725fe-7870-483c-b76b-74f4069f3386-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/344725fe-7870-483c-b76b-74f4069f3386-routing-overlay-volume\") pod \"344725fe-7870-483c-b76b-74f4069f3386\" (UID: \"344725fe-7870-483c-b76b-74f4069f3386\") "
Nov 04 02:27:44.675227 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.675194    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/344725fe-7870-483c-b76b-74f4069f3386-kube-api-access-q794r\" (UniqueName: \"kubernetes.io/projected/344725fe-7870-483c-b76b-74f4069f3386-kube-api-access-q794r\") pod \"344725fe-7870-483c-b76b-74f4069f3386\" (UID: \"344725fe-7870-483c-b76b-74f4069f3386\") "
Nov 04 02:27:44.675492 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.675346    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/344725fe-7870-483c-b76b-74f4069f3386-provider-credentials\" (UniqueName: \"kubernetes.io/projected/344725fe-7870-483c-b76b-74f4069f3386-provider-credentials\") pod \"344725fe-7870-483c-b76b-74f4069f3386\" (UID: \"344725fe-7870-483c-b76b-74f4069f3386\") "
Nov 04 02:27:44.675492 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.675383    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/344725fe-7870-483c-b76b-74f4069f3386-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/344725fe-7870-483c-b76b-74f4069f3386-plugins-config-volume\") pod \"344725fe-7870-483c-b76b-74f4069f3386\" (UID: \"344725fe-7870-483c-b76b-74f4069f3386\") "
Nov 04 02:27:44.675660 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.675632    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/344725fe-7870-483c-b76b-74f4069f3386-routing-overlay-volume" pod "344725fe-7870-483c-b76b-74f4069f3386" (UID: "344725fe-7870-483c-b76b-74f4069f3386"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:27:44.675993 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.675968    4865 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/742654db-eedb-4c8d-b61a-504ae2932fea-routing-overlay-volume\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:44.676081 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.676003    4865 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-provider-credentials\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:44.676081 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.676021    4865 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/344725fe-7870-483c-b76b-74f4069f3386-routing-overlay-volume\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:44.676081 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.676036    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tq7rg\" (UniqueName: \"kubernetes.io/projected/742654db-eedb-4c8d-b61a-504ae2932fea-kube-api-access-tq7rg\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:44.676081 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.676043    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/344725fe-7870-483c-b76b-74f4069f3386-plugins-config-volume" pod "344725fe-7870-483c-b76b-74f4069f3386" (UID: "344725fe-7870-483c-b76b-74f4069f3386"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:27:44.676081 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.676051    4865 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/742654db-eedb-4c8d-b61a-504ae2932fea-plugins-config-volume\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:44.678136 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.678109    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/344725fe-7870-483c-b76b-74f4069f3386-provider-credentials" pod "344725fe-7870-483c-b76b-74f4069f3386" (UID: "344725fe-7870-483c-b76b-74f4069f3386"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:27:44.678259 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.678225    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/344725fe-7870-483c-b76b-74f4069f3386-kube-api-access-q794r" pod "344725fe-7870-483c-b76b-74f4069f3386" (UID: "344725fe-7870-483c-b76b-74f4069f3386"). InnerVolumeSpecName "kube-api-access-q794r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:27:44.777714 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.777675    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q794r\" (UniqueName: \"kubernetes.io/projected/344725fe-7870-483c-b76b-74f4069f3386-kube-api-access-q794r\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:44.777714 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.777714    4865 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/344725fe-7870-483c-b76b-74f4069f3386-provider-credentials\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:44.777966 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.777739    4865 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/344725fe-7870-483c-b76b-74f4069f3386-plugins-config-volume\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:44.850365 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.850335    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"
Nov 04 02:27:44.850365 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.850349    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp" event={"ID":"742654db-eedb-4c8d-b61a-504ae2932fea","Type":"ContainerDied","Data":"481b7d89f2d65c1c67630cc42c02347cb0067c16bf2b467935bb7bff33b7fce2"}
Nov 04 02:27:44.850619 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.850417    4865 scope.go:122] "RemoveContainer" containerID="3b2bfc056ff4b0e23ad26ca200f6398d5cd15fc7398081328fdb6e4e19099887"
Nov 04 02:27:44.852007 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.851974    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h" event={"ID":"344725fe-7870-483c-b76b-74f4069f3386","Type":"ContainerDied","Data":"bf67c1809d7af4635a1e3d10dafca58dd8a7276f675169f036ac843fd3e7c641"}
Nov 04 02:27:44.852140 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.851990    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-68df898747-shn7h"
Nov 04 02:27:44.860242 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:44.860216    4865 scope.go:122] "RemoveContainer" containerID="3bea87ed0e2f4d441f8d3981548b538f38304b8cd8e71371d62ad67be665770d"
Nov 04 02:27:45.017865 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:45.017834    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-68df898747-shn7h"]
Nov 04 02:27:45.024837 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:45.024811    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-68df898747-shn7h"]
Nov 04 02:27:45.138937 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:45.138878    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"]
Nov 04 02:27:45.142190 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:45.142161    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp"]
Nov 04 02:27:46.895001 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:46.894968    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="344725fe-7870-483c-b76b-74f4069f3386" path="/var/lib/kubelet/pods/344725fe-7870-483c-b76b-74f4069f3386/volumes"
Nov 04 02:27:46.895416 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:46.895382    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="742654db-eedb-4c8d-b61a-504ae2932fea" path="/var/lib/kubelet/pods/742654db-eedb-4c8d-b61a-504ae2932fea/volumes"
Nov 04 02:27:57.318138 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.318101    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"]
Nov 04 02:27:57.318633 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.318336    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv" podUID="134d6dc7-0882-4fe3-af67-d7ac7dc05e82" containerName="maas-api" containerID="cri-o://eb45a9da2357ff57b92a696a76c4652dc3da7e68243c31b4ba4d2b009f2c9272" gracePeriod=30
Nov 04 02:27:57.611405 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.611373    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:27:57.727163 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.724472    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-openshift-service-ca\") pod \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\" (UID: \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\") "
Nov 04 02:27:57.727163 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.724540    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-kube-api-access-mf428\" (UniqueName: \"kubernetes.io/projected/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-kube-api-access-mf428\") pod \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\" (UID: \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\") "
Nov 04 02:27:57.727163 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.724678    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-maas-api-tls\") pod \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\" (UID: \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\") "
Nov 04 02:27:57.727163 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.724723    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-metrics-certs\" (UniqueName: \"kubernetes.io/secret/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-metrics-certs\") pod \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\" (UID: \"134d6dc7-0882-4fe3-af67-d7ac7dc05e82\") "
Nov 04 02:27:57.727163 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.726436    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-openshift-service-ca" pod "134d6dc7-0882-4fe3-af67-d7ac7dc05e82" (UID: "134d6dc7-0882-4fe3-af67-d7ac7dc05e82"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:27:57.736417 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.736362    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-metrics-certs" pod "134d6dc7-0882-4fe3-af67-d7ac7dc05e82" (UID: "134d6dc7-0882-4fe3-af67-d7ac7dc05e82"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:27:57.738721 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.738676    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-maas-api-tls" pod "134d6dc7-0882-4fe3-af67-d7ac7dc05e82" (UID: "134d6dc7-0882-4fe3-af67-d7ac7dc05e82"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:27:57.740503 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.740459    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-kube-api-access-mf428" pod "134d6dc7-0882-4fe3-af67-d7ac7dc05e82" (UID: "134d6dc7-0882-4fe3-af67-d7ac7dc05e82"). InnerVolumeSpecName "kube-api-access-mf428". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:27:57.826803 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.826766    4865 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-maas-api-tls\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:57.826803 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.826797    4865 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-metrics-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:57.826803 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.826808    4865 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-openshift-service-ca\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:57.827041 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.826819    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf428\" (UniqueName: \"kubernetes.io/projected/134d6dc7-0882-4fe3-af67-d7ac7dc05e82-kube-api-access-mf428\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:57.927293 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.927184    4865 generic.go:356] "Generic (PLEG): container finished" podID="134d6dc7-0882-4fe3-af67-d7ac7dc05e82" containerID="eb45a9da2357ff57b92a696a76c4652dc3da7e68243c31b4ba4d2b009f2c9272" exitCode=0
Nov 04 02:27:57.927293 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.927233    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"
Nov 04 02:27:57.927497 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.927236    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv" event={"ID":"134d6dc7-0882-4fe3-af67-d7ac7dc05e82","Type":"ContainerDied","Data":"eb45a9da2357ff57b92a696a76c4652dc3da7e68243c31b4ba4d2b009f2c9272"}
Nov 04 02:27:57.927497 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.927362    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv" event={"ID":"134d6dc7-0882-4fe3-af67-d7ac7dc05e82","Type":"ContainerDied","Data":"2046103f945b2d0fc5c8a518874e534576b50b0e0fa64dc2a17815b7f7d5ff6f"}
Nov 04 02:27:57.927497 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.927394    4865 scope.go:122] "RemoveContainer" containerID="eb45a9da2357ff57b92a696a76c4652dc3da7e68243c31b4ba4d2b009f2c9272"
Nov 04 02:27:57.937060 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.937041    4865 scope.go:122] "RemoveContainer" containerID="eb45a9da2357ff57b92a696a76c4652dc3da7e68243c31b4ba4d2b009f2c9272"
Nov 04 02:27:57.937400 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:27:57.937375    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb45a9da2357ff57b92a696a76c4652dc3da7e68243c31b4ba4d2b009f2c9272\": container with ID starting with eb45a9da2357ff57b92a696a76c4652dc3da7e68243c31b4ba4d2b009f2c9272 not found: ID does not exist" containerID="eb45a9da2357ff57b92a696a76c4652dc3da7e68243c31b4ba4d2b009f2c9272"
Nov 04 02:27:57.937471 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:57.937413    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb45a9da2357ff57b92a696a76c4652dc3da7e68243c31b4ba4d2b009f2c9272"} err="failed to get container status \"eb45a9da2357ff57b92a696a76c4652dc3da7e68243c31b4ba4d2b009f2c9272\": rpc error: code = NotFound desc = could not find container \"eb45a9da2357ff57b92a696a76c4652dc3da7e68243c31b4ba4d2b009f2c9272\": container with ID starting with eb45a9da2357ff57b92a696a76c4652dc3da7e68243c31b4ba4d2b009f2c9272 not found: ID does not exist"
Nov 04 02:27:58.100627 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:58.100593    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"]
Nov 04 02:27:58.106292 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:58.106256    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-dab533-6b4c88c677-rswdv"]
Nov 04 02:27:58.406669 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:27:58.406624    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:27:58.895207 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:27:58.895173    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="134d6dc7-0882-4fe3-af67-d7ac7dc05e82" path="/var/lib/kubelet/pods/134d6dc7-0882-4fe3-af67-d7ac7dc05e82/volumes"
Nov 04 02:28:07.198921 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:07.198876    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"]
Nov 04 02:28:07.199604 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:07.199111    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26" podUID="87f5ff3d-b612-4b82-a700-221881b22907" containerName="istio-proxy" containerID="cri-o://791b97814e4673b72b872c553f55f3c3c19109a79a525f7b9b86a3ded1889154" gracePeriod=30
Nov 04 02:28:12.327213 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.327181    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:28:12.454101 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.454073    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:28:12.529588 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.529521    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87f5ff3d-b612-4b82-a700-221881b22907-kube-api-access-jhdqf\" (UniqueName: \"kubernetes.io/projected/87f5ff3d-b612-4b82-a700-221881b22907-kube-api-access-jhdqf\") pod \"87f5ff3d-b612-4b82-a700-221881b22907\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") "
Nov 04 02:28:12.529724 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.529590    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-istio-data\") pod \"87f5ff3d-b612-4b82-a700-221881b22907\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") "
Nov 04 02:28:12.529724 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.529626    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-workload-socket\") pod \"87f5ff3d-b612-4b82-a700-221881b22907\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") "
Nov 04 02:28:12.529724 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.529677    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87f5ff3d-b612-4b82-a700-221881b22907-istio-token\" (UniqueName: \"kubernetes.io/projected/87f5ff3d-b612-4b82-a700-221881b22907-istio-token\") pod \"87f5ff3d-b612-4b82-a700-221881b22907\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") "
Nov 04 02:28:12.529868 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.529727    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/87f5ff3d-b612-4b82-a700-221881b22907-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87f5ff3d-b612-4b82-a700-221881b22907-istio-podinfo\") pod \"87f5ff3d-b612-4b82-a700-221881b22907\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") "
Nov 04 02:28:12.529868 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.529751    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-workload-certs\") pod \"87f5ff3d-b612-4b82-a700-221881b22907\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") "
Nov 04 02:28:12.529868 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.529785    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-istio-envoy\") pod \"87f5ff3d-b612-4b82-a700-221881b22907\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") "
Nov 04 02:28:12.529868 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.529820    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87f5ff3d-b612-4b82-a700-221881b22907-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87f5ff3d-b612-4b82-a700-221881b22907-istiod-ca-cert\") pod \"87f5ff3d-b612-4b82-a700-221881b22907\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") "
Nov 04 02:28:12.529868 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.529846    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-credential-socket\") pod \"87f5ff3d-b612-4b82-a700-221881b22907\" (UID: \"87f5ff3d-b612-4b82-a700-221881b22907\") "
Nov 04 02:28:12.530176 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.529921    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-istio-data" pod "87f5ff3d-b612-4b82-a700-221881b22907" (UID: "87f5ff3d-b612-4b82-a700-221881b22907"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:28:12.530176 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.530008    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-workload-socket" pod "87f5ff3d-b612-4b82-a700-221881b22907" (UID: "87f5ff3d-b612-4b82-a700-221881b22907"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:28:12.530176 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.530079    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-workload-certs" pod "87f5ff3d-b612-4b82-a700-221881b22907" (UID: "87f5ff3d-b612-4b82-a700-221881b22907"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:28:12.530357 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.530203    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-credential-socket" pod "87f5ff3d-b612-4b82-a700-221881b22907" (UID: "87f5ff3d-b612-4b82-a700-221881b22907"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:28:12.530357 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.530314    4865 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-istio-data\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:12.530357 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.530335    4865 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-workload-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:12.530357 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.530350    4865 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-workload-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:12.530569 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.530364    4865 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-credential-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:12.530569 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.530395    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87f5ff3d-b612-4b82-a700-221881b22907-istiod-ca-cert" pod "87f5ff3d-b612-4b82-a700-221881b22907" (UID: "87f5ff3d-b612-4b82-a700-221881b22907"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:28:12.531936 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.531906    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87f5ff3d-b612-4b82-a700-221881b22907-istio-token" pod "87f5ff3d-b612-4b82-a700-221881b22907" (UID: "87f5ff3d-b612-4b82-a700-221881b22907"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:28:12.532044 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.531939    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87f5ff3d-b612-4b82-a700-221881b22907-kube-api-access-jhdqf" pod "87f5ff3d-b612-4b82-a700-221881b22907" (UID: "87f5ff3d-b612-4b82-a700-221881b22907"). InnerVolumeSpecName "kube-api-access-jhdqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:28:12.532044 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.531948    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-istio-envoy" pod "87f5ff3d-b612-4b82-a700-221881b22907" (UID: "87f5ff3d-b612-4b82-a700-221881b22907"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:28:12.532044 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.531974    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/87f5ff3d-b612-4b82-a700-221881b22907-istio-podinfo" pod "87f5ff3d-b612-4b82-a700-221881b22907" (UID: "87f5ff3d-b612-4b82-a700-221881b22907"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:28:12.631427 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.631398    4865 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87f5ff3d-b612-4b82-a700-221881b22907-istio-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:12.631427 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.631424    4865 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87f5ff3d-b612-4b82-a700-221881b22907-istio-podinfo\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:12.631427 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.631435    4865 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87f5ff3d-b612-4b82-a700-221881b22907-istio-envoy\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:12.631727 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.631444    4865 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87f5ff3d-b612-4b82-a700-221881b22907-istiod-ca-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:12.631727 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:12.631454    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhdqf\" (UniqueName: \"kubernetes.io/projected/87f5ff3d-b612-4b82-a700-221881b22907-kube-api-access-jhdqf\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:13.008450 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:13.008411    4865 generic.go:356] "Generic (PLEG): container finished" podID="87f5ff3d-b612-4b82-a700-221881b22907" containerID="791b97814e4673b72b872c553f55f3c3c19109a79a525f7b9b86a3ded1889154" exitCode=0
Nov 04 02:28:13.008688 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:13.008468    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"
Nov 04 02:28:13.008688 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:13.008495    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26" event={"ID":"87f5ff3d-b612-4b82-a700-221881b22907","Type":"ContainerDied","Data":"791b97814e4673b72b872c553f55f3c3c19109a79a525f7b9b86a3ded1889154"}
Nov 04 02:28:13.008688 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:13.008557    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26" event={"ID":"87f5ff3d-b612-4b82-a700-221881b22907","Type":"ContainerDied","Data":"d636b67c4f377bea341cf1d268743ff77b8cb0ac86da674461f42e3f18ef9a78"}
Nov 04 02:28:13.008688 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:13.008590    4865 scope.go:122] "RemoveContainer" containerID="791b97814e4673b72b872c553f55f3c3c19109a79a525f7b9b86a3ded1889154"
Nov 04 02:28:13.017329 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:13.017309    4865 scope.go:122] "RemoveContainer" containerID="8d1b814eae74f3709e5a75fd6b63af31e1b508ead98fcb8f78fb1289afa18c41"
Nov 04 02:28:13.025626 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:13.025610    4865 scope.go:122] "RemoveContainer" containerID="791b97814e4673b72b872c553f55f3c3c19109a79a525f7b9b86a3ded1889154"
Nov 04 02:28:13.025889 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:28:13.025867    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"791b97814e4673b72b872c553f55f3c3c19109a79a525f7b9b86a3ded1889154\": container with ID starting with 791b97814e4673b72b872c553f55f3c3c19109a79a525f7b9b86a3ded1889154 not found: ID does not exist" containerID="791b97814e4673b72b872c553f55f3c3c19109a79a525f7b9b86a3ded1889154"
Nov 04 02:28:13.025935 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:13.025897    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"791b97814e4673b72b872c553f55f3c3c19109a79a525f7b9b86a3ded1889154"} err="failed to get container status \"791b97814e4673b72b872c553f55f3c3c19109a79a525f7b9b86a3ded1889154\": rpc error: code = NotFound desc = could not find container \"791b97814e4673b72b872c553f55f3c3c19109a79a525f7b9b86a3ded1889154\": container with ID starting with 791b97814e4673b72b872c553f55f3c3c19109a79a525f7b9b86a3ded1889154 not found: ID does not exist"
Nov 04 02:28:13.025935 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:13.025915    4865 scope.go:122] "RemoveContainer" containerID="8d1b814eae74f3709e5a75fd6b63af31e1b508ead98fcb8f78fb1289afa18c41"
Nov 04 02:28:13.026142 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:28:13.026124    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d1b814eae74f3709e5a75fd6b63af31e1b508ead98fcb8f78fb1289afa18c41\": container with ID starting with 8d1b814eae74f3709e5a75fd6b63af31e1b508ead98fcb8f78fb1289afa18c41 not found: ID does not exist" containerID="8d1b814eae74f3709e5a75fd6b63af31e1b508ead98fcb8f78fb1289afa18c41"
Nov 04 02:28:13.026182 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:13.026150    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d1b814eae74f3709e5a75fd6b63af31e1b508ead98fcb8f78fb1289afa18c41"} err="failed to get container status \"8d1b814eae74f3709e5a75fd6b63af31e1b508ead98fcb8f78fb1289afa18c41\": rpc error: code = NotFound desc = could not find container \"8d1b814eae74f3709e5a75fd6b63af31e1b508ead98fcb8f78fb1289afa18c41\": container with ID starting with 8d1b814eae74f3709e5a75fd6b63af31e1b508ead98fcb8f78fb1289afa18c41 not found: ID does not exist"
Nov 04 02:28:13.177677 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:13.177590    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"]
Nov 04 02:28:13.228409 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:13.228374    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26"]
Nov 04 02:28:13.707087 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:13.707052    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"]
Nov 04 02:28:13.707596 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:13.707252    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh" podUID="0bd330e6-5c2c-4095-b862-4e13cec905b4" containerName="maas-api" containerID="cri-o://bfb3230f78479833aaa08ce55bbb79ff76aa0a2be7861a2adf8aa8c0aac9922a" gracePeriod=30
Nov 04 02:28:13.992170 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:13.992138    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:28:14.014849 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.014810    4865 generic.go:356] "Generic (PLEG): container finished" podID="0bd330e6-5c2c-4095-b862-4e13cec905b4" containerID="bfb3230f78479833aaa08ce55bbb79ff76aa0a2be7861a2adf8aa8c0aac9922a" exitCode=0
Nov 04 02:28:14.015034 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.014862    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"
Nov 04 02:28:14.015034 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.014893    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh" event={"ID":"0bd330e6-5c2c-4095-b862-4e13cec905b4","Type":"ContainerDied","Data":"bfb3230f78479833aaa08ce55bbb79ff76aa0a2be7861a2adf8aa8c0aac9922a"}
Nov 04 02:28:14.015034 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.014949    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh" event={"ID":"0bd330e6-5c2c-4095-b862-4e13cec905b4","Type":"ContainerDied","Data":"deccff5f07ba54910101788d355a28d7eb4e57a67295febbfc4d1fc02fd34cb5"}
Nov 04 02:28:14.015034 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.014983    4865 scope.go:122] "RemoveContainer" containerID="bfb3230f78479833aaa08ce55bbb79ff76aa0a2be7861a2adf8aa8c0aac9922a"
Nov 04 02:28:14.023792 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.023774    4865 scope.go:122] "RemoveContainer" containerID="bfb3230f78479833aaa08ce55bbb79ff76aa0a2be7861a2adf8aa8c0aac9922a"
Nov 04 02:28:14.024040 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:28:14.024022    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfb3230f78479833aaa08ce55bbb79ff76aa0a2be7861a2adf8aa8c0aac9922a\": container with ID starting with bfb3230f78479833aaa08ce55bbb79ff76aa0a2be7861a2adf8aa8c0aac9922a not found: ID does not exist" containerID="bfb3230f78479833aaa08ce55bbb79ff76aa0a2be7861a2adf8aa8c0aac9922a"
Nov 04 02:28:14.024131 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.024050    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfb3230f78479833aaa08ce55bbb79ff76aa0a2be7861a2adf8aa8c0aac9922a"} err="failed to get container status \"bfb3230f78479833aaa08ce55bbb79ff76aa0a2be7861a2adf8aa8c0aac9922a\": rpc error: code = NotFound desc = could not find container \"bfb3230f78479833aaa08ce55bbb79ff76aa0a2be7861a2adf8aa8c0aac9922a\": container with ID starting with bfb3230f78479833aaa08ce55bbb79ff76aa0a2be7861a2adf8aa8c0aac9922a not found: ID does not exist"
Nov 04 02:28:14.051989 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.051961    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0bd330e6-5c2c-4095-b862-4e13cec905b4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bd330e6-5c2c-4095-b862-4e13cec905b4-openshift-service-ca\") pod \"0bd330e6-5c2c-4095-b862-4e13cec905b4\" (UID: \"0bd330e6-5c2c-4095-b862-4e13cec905b4\") "
Nov 04 02:28:14.052121 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.052049    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0bd330e6-5c2c-4095-b862-4e13cec905b4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bd330e6-5c2c-4095-b862-4e13cec905b4-maas-api-tls\") pod \"0bd330e6-5c2c-4095-b862-4e13cec905b4\" (UID: \"0bd330e6-5c2c-4095-b862-4e13cec905b4\") "
Nov 04 02:28:14.052121 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.052098    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0bd330e6-5c2c-4095-b862-4e13cec905b4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bd330e6-5c2c-4095-b862-4e13cec905b4-metrics-certs\") pod \"0bd330e6-5c2c-4095-b862-4e13cec905b4\" (UID: \"0bd330e6-5c2c-4095-b862-4e13cec905b4\") "
Nov 04 02:28:14.052203 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.052134    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0bd330e6-5c2c-4095-b862-4e13cec905b4-kube-api-access-b6spn\" (UniqueName: \"kubernetes.io/projected/0bd330e6-5c2c-4095-b862-4e13cec905b4-kube-api-access-b6spn\") pod \"0bd330e6-5c2c-4095-b862-4e13cec905b4\" (UID: \"0bd330e6-5c2c-4095-b862-4e13cec905b4\") "
Nov 04 02:28:14.052345 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.052324    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bd330e6-5c2c-4095-b862-4e13cec905b4-openshift-service-ca" pod "0bd330e6-5c2c-4095-b862-4e13cec905b4" (UID: "0bd330e6-5c2c-4095-b862-4e13cec905b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:28:14.052473 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.052449    4865 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bd330e6-5c2c-4095-b862-4e13cec905b4-openshift-service-ca\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:14.054269 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.054239    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bd330e6-5c2c-4095-b862-4e13cec905b4-maas-api-tls" pod "0bd330e6-5c2c-4095-b862-4e13cec905b4" (UID: "0bd330e6-5c2c-4095-b862-4e13cec905b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:28:14.054269 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.054259    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bd330e6-5c2c-4095-b862-4e13cec905b4-metrics-certs" pod "0bd330e6-5c2c-4095-b862-4e13cec905b4" (UID: "0bd330e6-5c2c-4095-b862-4e13cec905b4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:28:14.054413 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.054307    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bd330e6-5c2c-4095-b862-4e13cec905b4-kube-api-access-b6spn" pod "0bd330e6-5c2c-4095-b862-4e13cec905b4" (UID: "0bd330e6-5c2c-4095-b862-4e13cec905b4"). InnerVolumeSpecName "kube-api-access-b6spn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:28:14.154078 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.154035    4865 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bd330e6-5c2c-4095-b862-4e13cec905b4-maas-api-tls\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:14.154078 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.154068    4865 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bd330e6-5c2c-4095-b862-4e13cec905b4-metrics-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:14.154078 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.154083    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6spn\" (UniqueName: \"kubernetes.io/projected/0bd330e6-5c2c-4095-b862-4e13cec905b4-kube-api-access-b6spn\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:14.439037 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.439004    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"]
Nov 04 02:28:14.443334 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.443307    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-3d082d-65cccb867f-c7nhh"]
Nov 04 02:28:14.896257 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.896225    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0bd330e6-5c2c-4095-b862-4e13cec905b4" path="/var/lib/kubelet/pods/0bd330e6-5c2c-4095-b862-4e13cec905b4/volumes"
Nov 04 02:28:14.896715 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:14.896699    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="87f5ff3d-b612-4b82-a700-221881b22907" path="/var/lib/kubelet/pods/87f5ff3d-b612-4b82-a700-221881b22907/volumes"
Nov 04 02:28:22.272323 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:22.272259    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"]
Nov 04 02:28:22.272759 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:22.272484    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd" podUID="707f17bc-7327-4cd7-9796-497331e55d9c" containerName="istio-proxy" containerID="cri-o://4041240c8fd95cd9b23a4ebb2fdc399e94e127ff3e4209654e0d361addf032e7" gracePeriod=30
Nov 04 02:28:23.352405 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352366    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr"]
Nov 04 02:28:23.352784 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352775    4865 cpu_manager.go:414] "containerMap: removing container" podUID="0bd330e6-5c2c-4095-b862-4e13cec905b4" containerName="maas-api"
Nov 04 02:28:23.352836 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352788    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0bd330e6-5c2c-4095-b862-4e13cec905b4" containerName="maas-api"
Nov 04 02:28:23.352836 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352798    4865 cpu_manager.go:414] "containerMap: removing container" podUID="134d6dc7-0882-4fe3-af67-d7ac7dc05e82" containerName="maas-api"
Nov 04 02:28:23.352836 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352804    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="134d6dc7-0882-4fe3-af67-d7ac7dc05e82" containerName="maas-api"
Nov 04 02:28:23.352836 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352817    4865 cpu_manager.go:414] "containerMap: removing container" podUID="42661d7a-b751-4a93-b66c-25c7d0275abf" containerName="istio-proxy"
Nov 04 02:28:23.352836 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352822    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42661d7a-b751-4a93-b66c-25c7d0275abf" containerName="istio-proxy"
Nov 04 02:28:23.352836 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352836    4865 cpu_manager.go:414] "containerMap: removing container" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352841    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352849    4865 cpu_manager.go:414] "containerMap: removing container" podUID="87f5ff3d-b612-4b82-a700-221881b22907" containerName="istio-proxy"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352853    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87f5ff3d-b612-4b82-a700-221881b22907" containerName="istio-proxy"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352870    4865 cpu_manager.go:414] "containerMap: removing container" podUID="42661d7a-b751-4a93-b66c-25c7d0275abf" containerName="istio-proxy"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352875    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42661d7a-b751-4a93-b66c-25c7d0275abf" containerName="istio-proxy"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352883    4865 cpu_manager.go:414] "containerMap: removing container" podUID="742654db-eedb-4c8d-b61a-504ae2932fea" containerName="payload-processing"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352889    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="742654db-eedb-4c8d-b61a-504ae2932fea" containerName="payload-processing"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352896    4865 cpu_manager.go:414] "containerMap: removing container" podUID="c22a4702-2ab7-4eb4-bb1f-64ee45937a41" containerName="istio-proxy"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352901    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c22a4702-2ab7-4eb4-bb1f-64ee45937a41" containerName="istio-proxy"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352908    4865 cpu_manager.go:414] "containerMap: removing container" podUID="87f5ff3d-b612-4b82-a700-221881b22907" containerName="istio-proxy"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352912    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87f5ff3d-b612-4b82-a700-221881b22907" containerName="istio-proxy"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352973    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="87f5ff3d-b612-4b82-a700-221881b22907" containerName="istio-proxy"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352982    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="42661d7a-b751-4a93-b66c-25c7d0275abf" containerName="istio-proxy"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352988    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="742654db-eedb-4c8d-b61a-504ae2932fea" containerName="payload-processing"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352993    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="9049038a-28f6-48ac-9971-a60bfe207660" containerName="istio-proxy"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.352999    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="c22a4702-2ab7-4eb4-bb1f-64ee45937a41" containerName="istio-proxy"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.353006    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="0bd330e6-5c2c-4095-b862-4e13cec905b4" containerName="maas-api"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.353012    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="134d6dc7-0882-4fe3-af67-d7ac7dc05e82" containerName="maas-api"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.353019    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="42661d7a-b751-4a93-b66c-25c7d0275abf" containerName="istio-proxy"
Nov 04 02:28:23.353020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.353026    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="344725fe-7870-483c-b76b-74f4069f3386" containerName="payload-processing"
Nov 04 02:28:23.357502 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.357486    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr"
Nov 04 02:28:23.369541 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.369517    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jjbr\" (UniqueName: \"kubernetes.io/projected/781ba848-3afb-4372-87dd-70d5279f2f2e-kube-api-access-9jjbr\") pod \"maas-api-revoke-sub-4ff54f84-vzcxr\" (UID: \"781ba848-3afb-4372-87dd-70d5279f2f2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr"
Nov 04 02:28:23.369664 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.369558    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/781ba848-3afb-4372-87dd-70d5279f2f2e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4ff54f84-vzcxr\" (UID: \"781ba848-3afb-4372-87dd-70d5279f2f2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr"
Nov 04 02:28:23.369664 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.369632    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/781ba848-3afb-4372-87dd-70d5279f2f2e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4ff54f84-vzcxr\" (UID: \"781ba848-3afb-4372-87dd-70d5279f2f2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr"
Nov 04 02:28:23.421213 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.418703    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr"]
Nov 04 02:28:23.462202 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.462167    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 02:28:23.462202 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.462167    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:28:23.462202 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.462201    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 02:28:23.462469 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.462207    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 02:28:23.470740 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.470708    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jjbr\" (UniqueName: \"kubernetes.io/projected/781ba848-3afb-4372-87dd-70d5279f2f2e-kube-api-access-9jjbr\") pod \"maas-api-revoke-sub-4ff54f84-vzcxr\" (UID: \"781ba848-3afb-4372-87dd-70d5279f2f2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr"
Nov 04 02:28:23.470841 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.470750    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/781ba848-3afb-4372-87dd-70d5279f2f2e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4ff54f84-vzcxr\" (UID: \"781ba848-3afb-4372-87dd-70d5279f2f2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr"
Nov 04 02:28:23.470841 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.470788    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/781ba848-3afb-4372-87dd-70d5279f2f2e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4ff54f84-vzcxr\" (UID: \"781ba848-3afb-4372-87dd-70d5279f2f2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr"
Nov 04 02:28:23.471512 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.471494    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/781ba848-3afb-4372-87dd-70d5279f2f2e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4ff54f84-vzcxr\" (UID: \"781ba848-3afb-4372-87dd-70d5279f2f2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr"
Nov 04 02:28:23.473144 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.473127    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/781ba848-3afb-4372-87dd-70d5279f2f2e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4ff54f84-vzcxr\" (UID: \"781ba848-3afb-4372-87dd-70d5279f2f2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr"
Nov 04 02:28:23.532545 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.532512    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jjbr\" (UniqueName: \"kubernetes.io/projected/781ba848-3afb-4372-87dd-70d5279f2f2e-kube-api-access-9jjbr\") pod \"maas-api-revoke-sub-4ff54f84-vzcxr\" (UID: \"781ba848-3afb-4372-87dd-70d5279f2f2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr"
Nov 04 02:28:23.670009 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.669928    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr"
Nov 04 02:28:23.671167 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.671132    4865 patch_prober.go:29] interesting pod/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 02:28:23.671300 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:23.671201    4865 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd" podUID="707f17bc-7327-4cd7-9796-497331e55d9c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 02:28:24.042903 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:24.042874    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr"]
Nov 04 02:28:24.044601 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:28:24.044575    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod781ba848_3afb_4372_87dd_70d5279f2f2e.slice/crio-5d193c9b9bee640d34e5cb23cb1be3d90d2be5d6024a181435c00cdf551a362a WatchSource:0}: Error finding container 5d193c9b9bee640d34e5cb23cb1be3d90d2be5d6024a181435c00cdf551a362a: Status 404 returned error can't find the container with id 5d193c9b9bee640d34e5cb23cb1be3d90d2be5d6024a181435c00cdf551a362a
Nov 04 02:28:24.066825 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:24.066795    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr" event={"ID":"781ba848-3afb-4372-87dd-70d5279f2f2e","Type":"ContainerStarted","Data":"5d193c9b9bee640d34e5cb23cb1be3d90d2be5d6024a181435c00cdf551a362a"}
Nov 04 02:28:25.073634 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:25.073595    4865 generic.go:356] "Generic (PLEG): container finished" podID="781ba848-3afb-4372-87dd-70d5279f2f2e" containerID="3e5ded182c946415b6ea74b3d9e6a193d47fb08d30eaf5b000761a2b5aace6ca" exitCode=0
Nov 04 02:28:25.074018 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:25.073639    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr" event={"ID":"781ba848-3afb-4372-87dd-70d5279f2f2e","Type":"ContainerDied","Data":"3e5ded182c946415b6ea74b3d9e6a193d47fb08d30eaf5b000761a2b5aace6ca"}
Nov 04 02:28:26.211899 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:26.211869    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr"
Nov 04 02:28:26.296295 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:26.296253    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/781ba848-3afb-4372-87dd-70d5279f2f2e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/781ba848-3afb-4372-87dd-70d5279f2f2e-maas-api-xks-ca\") pod \"781ba848-3afb-4372-87dd-70d5279f2f2e\" (UID: \"781ba848-3afb-4372-87dd-70d5279f2f2e\") "
Nov 04 02:28:26.296460 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:26.296354    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/781ba848-3afb-4372-87dd-70d5279f2f2e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/781ba848-3afb-4372-87dd-70d5279f2f2e-maas-api-openshift-ca\") pod \"781ba848-3afb-4372-87dd-70d5279f2f2e\" (UID: \"781ba848-3afb-4372-87dd-70d5279f2f2e\") "
Nov 04 02:28:26.296460 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:26.296430    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/781ba848-3afb-4372-87dd-70d5279f2f2e-kube-api-access-9jjbr\" (UniqueName: \"kubernetes.io/projected/781ba848-3afb-4372-87dd-70d5279f2f2e-kube-api-access-9jjbr\") pod \"781ba848-3afb-4372-87dd-70d5279f2f2e\" (UID: \"781ba848-3afb-4372-87dd-70d5279f2f2e\") "
Nov 04 02:28:26.296704 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:26.296683    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/781ba848-3afb-4372-87dd-70d5279f2f2e-maas-api-openshift-ca" pod "781ba848-3afb-4372-87dd-70d5279f2f2e" (UID: "781ba848-3afb-4372-87dd-70d5279f2f2e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:28:26.296831 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:26.296815    4865 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/781ba848-3afb-4372-87dd-70d5279f2f2e-maas-api-openshift-ca\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:26.298502 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:26.298479    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/781ba848-3afb-4372-87dd-70d5279f2f2e-maas-api-xks-ca" pod "781ba848-3afb-4372-87dd-70d5279f2f2e" (UID: "781ba848-3afb-4372-87dd-70d5279f2f2e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:28:26.298589 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:26.298563    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/781ba848-3afb-4372-87dd-70d5279f2f2e-kube-api-access-9jjbr" pod "781ba848-3afb-4372-87dd-70d5279f2f2e" (UID: "781ba848-3afb-4372-87dd-70d5279f2f2e"). InnerVolumeSpecName "kube-api-access-9jjbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:28:26.398242 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:26.398166    4865 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/781ba848-3afb-4372-87dd-70d5279f2f2e-maas-api-xks-ca\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:26.398242 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:26.398196    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jjbr\" (UniqueName: \"kubernetes.io/projected/781ba848-3afb-4372-87dd-70d5279f2f2e-kube-api-access-9jjbr\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:26.959590 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:26.959557    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:28:27.084813 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.084785    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr"
Nov 04 02:28:27.084813 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.084807    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4ff54f84-vzcxr" event={"ID":"781ba848-3afb-4372-87dd-70d5279f2f2e","Type":"ContainerDied","Data":"5d193c9b9bee640d34e5cb23cb1be3d90d2be5d6024a181435c00cdf551a362a"}
Nov 04 02:28:27.085010 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.084849    4865 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5d193c9b9bee640d34e5cb23cb1be3d90d2be5d6024a181435c00cdf551a362a"
Nov 04 02:28:27.537326 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.537248    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:28:27.609063 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609032    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-credential-socket\") pod \"707f17bc-7327-4cd7-9796-497331e55d9c\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") "
Nov 04 02:28:27.609240 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609081    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-workload-certs\") pod \"707f17bc-7327-4cd7-9796-497331e55d9c\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") "
Nov 04 02:28:27.609240 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609118    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/707f17bc-7327-4cd7-9796-497331e55d9c-istio-token\" (UniqueName: \"kubernetes.io/projected/707f17bc-7327-4cd7-9796-497331e55d9c-istio-token\") pod \"707f17bc-7327-4cd7-9796-497331e55d9c\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") "
Nov 04 02:28:27.609240 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609151    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/707f17bc-7327-4cd7-9796-497331e55d9c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/707f17bc-7327-4cd7-9796-497331e55d9c-istiod-ca-cert\") pod \"707f17bc-7327-4cd7-9796-497331e55d9c\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") "
Nov 04 02:28:27.609240 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609197    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-istio-data\") pod \"707f17bc-7327-4cd7-9796-497331e55d9c\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") "
Nov 04 02:28:27.609240 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609229    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/707f17bc-7327-4cd7-9796-497331e55d9c-kube-api-access-66zfq\" (UniqueName: \"kubernetes.io/projected/707f17bc-7327-4cd7-9796-497331e55d9c-kube-api-access-66zfq\") pod \"707f17bc-7327-4cd7-9796-497331e55d9c\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") "
Nov 04 02:28:27.609544 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609332    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/707f17bc-7327-4cd7-9796-497331e55d9c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/707f17bc-7327-4cd7-9796-497331e55d9c-istio-podinfo\") pod \"707f17bc-7327-4cd7-9796-497331e55d9c\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") "
Nov 04 02:28:27.609544 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609368    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-istio-envoy\") pod \"707f17bc-7327-4cd7-9796-497331e55d9c\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") "
Nov 04 02:28:27.609544 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609384    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-credential-socket" pod "707f17bc-7327-4cd7-9796-497331e55d9c" (UID: "707f17bc-7327-4cd7-9796-497331e55d9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:28:27.609544 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609419    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-workload-socket\") pod \"707f17bc-7327-4cd7-9796-497331e55d9c\" (UID: \"707f17bc-7327-4cd7-9796-497331e55d9c\") "
Nov 04 02:28:27.609544 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609519    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-workload-certs" pod "707f17bc-7327-4cd7-9796-497331e55d9c" (UID: "707f17bc-7327-4cd7-9796-497331e55d9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:28:27.609877 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609606    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-istio-data" pod "707f17bc-7327-4cd7-9796-497331e55d9c" (UID: "707f17bc-7327-4cd7-9796-497331e55d9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:28:27.609877 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609613    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/707f17bc-7327-4cd7-9796-497331e55d9c-istiod-ca-cert" pod "707f17bc-7327-4cd7-9796-497331e55d9c" (UID: "707f17bc-7327-4cd7-9796-497331e55d9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:28:27.609877 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609780    4865 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-istio-data\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:27.609877 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609793    4865 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-credential-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:27.609877 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609806    4865 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-workload-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:27.609877 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609819    4865 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/707f17bc-7327-4cd7-9796-497331e55d9c-istiod-ca-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:27.609877 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.609820    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-workload-socket" pod "707f17bc-7327-4cd7-9796-497331e55d9c" (UID: "707f17bc-7327-4cd7-9796-497331e55d9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:28:27.611619 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.611589    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/707f17bc-7327-4cd7-9796-497331e55d9c-kube-api-access-66zfq" pod "707f17bc-7327-4cd7-9796-497331e55d9c" (UID: "707f17bc-7327-4cd7-9796-497331e55d9c"). InnerVolumeSpecName "kube-api-access-66zfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:28:27.611724 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.611695    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-istio-envoy" pod "707f17bc-7327-4cd7-9796-497331e55d9c" (UID: "707f17bc-7327-4cd7-9796-497331e55d9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:28:27.611824 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.611809    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/707f17bc-7327-4cd7-9796-497331e55d9c-istio-token" pod "707f17bc-7327-4cd7-9796-497331e55d9c" (UID: "707f17bc-7327-4cd7-9796-497331e55d9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:28:27.611867 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.611831    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/707f17bc-7327-4cd7-9796-497331e55d9c-istio-podinfo" pod "707f17bc-7327-4cd7-9796-497331e55d9c" (UID: "707f17bc-7327-4cd7-9796-497331e55d9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:28:27.711416 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.711384    4865 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-istio-envoy\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:27.711416 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.711412    4865 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/707f17bc-7327-4cd7-9796-497331e55d9c-workload-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:27.711416 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.711422    4865 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/707f17bc-7327-4cd7-9796-497331e55d9c-istio-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:27.711647 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.711432    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66zfq\" (UniqueName: \"kubernetes.io/projected/707f17bc-7327-4cd7-9796-497331e55d9c-kube-api-access-66zfq\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:27.711647 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:27.711441    4865 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/707f17bc-7327-4cd7-9796-497331e55d9c-istio-podinfo\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:28.091468 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:28.091432    4865 generic.go:356] "Generic (PLEG): container finished" podID="707f17bc-7327-4cd7-9796-497331e55d9c" containerID="4041240c8fd95cd9b23a4ebb2fdc399e94e127ff3e4209654e0d361addf032e7" exitCode=0
Nov 04 02:28:28.091628 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:28.091483    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"
Nov 04 02:28:28.091628 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:28.091500    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd" event={"ID":"707f17bc-7327-4cd7-9796-497331e55d9c","Type":"ContainerDied","Data":"4041240c8fd95cd9b23a4ebb2fdc399e94e127ff3e4209654e0d361addf032e7"}
Nov 04 02:28:28.091628 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:28.091541    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd" event={"ID":"707f17bc-7327-4cd7-9796-497331e55d9c","Type":"ContainerDied","Data":"8b50128d9c8497d1c33975a1f0efc2eeefe18d961650a1c6abe1e7923c9b7471"}
Nov 04 02:28:28.091628 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:28.091565    4865 scope.go:122] "RemoveContainer" containerID="4041240c8fd95cd9b23a4ebb2fdc399e94e127ff3e4209654e0d361addf032e7"
Nov 04 02:28:28.106796 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:28.106772    4865 scope.go:122] "RemoveContainer" containerID="d1e597d977231f66f057dc7f061c3ef64c330640ccbab84515f07b9b7b868170"
Nov 04 02:28:28.115073 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:28.115043    4865 scope.go:122] "RemoveContainer" containerID="4041240c8fd95cd9b23a4ebb2fdc399e94e127ff3e4209654e0d361addf032e7"
Nov 04 02:28:28.115496 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:28:28.115459    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4041240c8fd95cd9b23a4ebb2fdc399e94e127ff3e4209654e0d361addf032e7\": container with ID starting with 4041240c8fd95cd9b23a4ebb2fdc399e94e127ff3e4209654e0d361addf032e7 not found: ID does not exist" containerID="4041240c8fd95cd9b23a4ebb2fdc399e94e127ff3e4209654e0d361addf032e7"
Nov 04 02:28:28.115581 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:28.115512    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4041240c8fd95cd9b23a4ebb2fdc399e94e127ff3e4209654e0d361addf032e7"} err="failed to get container status \"4041240c8fd95cd9b23a4ebb2fdc399e94e127ff3e4209654e0d361addf032e7\": rpc error: code = NotFound desc = could not find container \"4041240c8fd95cd9b23a4ebb2fdc399e94e127ff3e4209654e0d361addf032e7\": container with ID starting with 4041240c8fd95cd9b23a4ebb2fdc399e94e127ff3e4209654e0d361addf032e7 not found: ID does not exist"
Nov 04 02:28:28.115581 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:28.115554    4865 scope.go:122] "RemoveContainer" containerID="d1e597d977231f66f057dc7f061c3ef64c330640ccbab84515f07b9b7b868170"
Nov 04 02:28:28.115899 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:28:28.115873    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1e597d977231f66f057dc7f061c3ef64c330640ccbab84515f07b9b7b868170\": container with ID starting with d1e597d977231f66f057dc7f061c3ef64c330640ccbab84515f07b9b7b868170 not found: ID does not exist" containerID="d1e597d977231f66f057dc7f061c3ef64c330640ccbab84515f07b9b7b868170"
Nov 04 02:28:28.115973 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:28.115908    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1e597d977231f66f057dc7f061c3ef64c330640ccbab84515f07b9b7b868170"} err="failed to get container status \"d1e597d977231f66f057dc7f061c3ef64c330640ccbab84515f07b9b7b868170\": rpc error: code = NotFound desc = could not find container \"d1e597d977231f66f057dc7f061c3ef64c330640ccbab84515f07b9b7b868170\": container with ID starting with d1e597d977231f66f057dc7f061c3ef64c330640ccbab84515f07b9b7b868170 not found: ID does not exist"
Nov 04 02:28:28.259410 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:28.259382    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"]
Nov 04 02:28:28.310123 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:28.310092    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd"]
Nov 04 02:28:28.403941 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:28:28.403857    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:28:28.896091 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:28.896054    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="707f17bc-7327-4cd7-9796-497331e55d9c" path="/var/lib/kubelet/pods/707f17bc-7327-4cd7-9796-497331e55d9c/volumes"
Nov 04 02:28:35.718923 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.718887    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d"]
Nov 04 02:28:35.719369 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.719336    4865 cpu_manager.go:414] "containerMap: removing container" podUID="707f17bc-7327-4cd7-9796-497331e55d9c" containerName="istio-proxy"
Nov 04 02:28:35.719369 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.719349    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="707f17bc-7327-4cd7-9796-497331e55d9c" containerName="istio-proxy"
Nov 04 02:28:35.719369 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.719367    4865 cpu_manager.go:414] "containerMap: removing container" podUID="707f17bc-7327-4cd7-9796-497331e55d9c" containerName="istio-proxy"
Nov 04 02:28:35.719369 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.719372    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="707f17bc-7327-4cd7-9796-497331e55d9c" containerName="istio-proxy"
Nov 04 02:28:35.719521 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.719382    4865 cpu_manager.go:414] "containerMap: removing container" podUID="781ba848-3afb-4372-87dd-70d5279f2f2e" containerName="revoke-keys"
Nov 04 02:28:35.719521 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.719389    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="781ba848-3afb-4372-87dd-70d5279f2f2e" containerName="revoke-keys"
Nov 04 02:28:35.719521 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.719458    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="707f17bc-7327-4cd7-9796-497331e55d9c" containerName="istio-proxy"
Nov 04 02:28:35.719521 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.719466    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="707f17bc-7327-4cd7-9796-497331e55d9c" containerName="istio-proxy"
Nov 04 02:28:35.719521 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.719473    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="781ba848-3afb-4372-87dd-70d5279f2f2e" containerName="revoke-keys"
Nov 04 02:28:35.719521 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.719480    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="87f5ff3d-b612-4b82-a700-221881b22907" containerName="istio-proxy"
Nov 04 02:28:35.724034 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.724014    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d"
Nov 04 02:28:35.778534 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.778499    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 02:28:35.817071 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.817040    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-01874c89-jkc6d\" (UID: \"db5f36dd-8c7f-441f-aaae-dd72fb1532ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d"
Nov 04 02:28:35.817255 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.817083    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t97xp\" (UniqueName: \"kubernetes.io/projected/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-kube-api-access-t97xp\") pod \"maas-api-revoke-sub-01874c89-jkc6d\" (UID: \"db5f36dd-8c7f-441f-aaae-dd72fb1532ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d"
Nov 04 02:28:35.817255 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.817183    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-01874c89-jkc6d\" (UID: \"db5f36dd-8c7f-441f-aaae-dd72fb1532ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d"
Nov 04 02:28:35.828946 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.828909    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:28:35.828946 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.828920    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 02:28:35.828946 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.828921    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 02:28:35.830872 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.830851    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d"]
Nov 04 02:28:35.918643 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.918605    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-01874c89-jkc6d\" (UID: \"db5f36dd-8c7f-441f-aaae-dd72fb1532ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d"
Nov 04 02:28:35.918643 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.918649    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t97xp\" (UniqueName: \"kubernetes.io/projected/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-kube-api-access-t97xp\") pod \"maas-api-revoke-sub-01874c89-jkc6d\" (UID: \"db5f36dd-8c7f-441f-aaae-dd72fb1532ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d"
Nov 04 02:28:35.918907 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.918685    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-01874c89-jkc6d\" (UID: \"db5f36dd-8c7f-441f-aaae-dd72fb1532ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d"
Nov 04 02:28:35.919349 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.919325    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-01874c89-jkc6d\" (UID: \"db5f36dd-8c7f-441f-aaae-dd72fb1532ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d"
Nov 04 02:28:35.920957 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.920937    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-01874c89-jkc6d\" (UID: \"db5f36dd-8c7f-441f-aaae-dd72fb1532ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d"
Nov 04 02:28:35.981132 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:35.981042    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t97xp\" (UniqueName: \"kubernetes.io/projected/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-kube-api-access-t97xp\") pod \"maas-api-revoke-sub-01874c89-jkc6d\" (UID: \"db5f36dd-8c7f-441f-aaae-dd72fb1532ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d"
Nov 04 02:28:36.036268 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:36.036234    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d"
Nov 04 02:28:36.409547 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:36.409514    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d"]
Nov 04 02:28:36.411367 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:28:36.411342    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb5f36dd_8c7f_441f_aaae_dd72fb1532ec.slice/crio-c6ea92eafa8f875ddc11ebda3dec69a65fcfb0685021e6c4924f4a509732fcff WatchSource:0}: Error finding container c6ea92eafa8f875ddc11ebda3dec69a65fcfb0685021e6c4924f4a509732fcff: Status 404 returned error can't find the container with id c6ea92eafa8f875ddc11ebda3dec69a65fcfb0685021e6c4924f4a509732fcff
Nov 04 02:28:37.139799 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:37.139700    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d" event={"ID":"db5f36dd-8c7f-441f-aaae-dd72fb1532ec","Type":"ContainerStarted","Data":"29264f86d9d08b1255949cb5527d7de866f45c71ccedf49bafe2ed909f38bf2a"}
Nov 04 02:28:37.139799 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:37.139763    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d" event={"ID":"db5f36dd-8c7f-441f-aaae-dd72fb1532ec","Type":"ContainerStarted","Data":"c6ea92eafa8f875ddc11ebda3dec69a65fcfb0685021e6c4924f4a509732fcff"}
Nov 04 02:28:37.257970 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:37.257920    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d" podStartSLOduration=2.2579053079999998 podStartE2EDuration="2.257905308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:28:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:28:37.256815944 +0000 UTC m=+1980.483602633" watchObservedRunningTime="2026-11-04 02:28:37.257905308 +0000 UTC m=+1980.484691976"
Nov 04 02:28:38.150578 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:38.150535    4865 generic.go:356] "Generic (PLEG): container finished" podID="db5f36dd-8c7f-441f-aaae-dd72fb1532ec" containerID="29264f86d9d08b1255949cb5527d7de866f45c71ccedf49bafe2ed909f38bf2a" exitCode=0
Nov 04 02:28:38.151033 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:38.150591    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d" event={"ID":"db5f36dd-8c7f-441f-aaae-dd72fb1532ec","Type":"ContainerDied","Data":"29264f86d9d08b1255949cb5527d7de866f45c71ccedf49bafe2ed909f38bf2a"}
Nov 04 02:28:39.286751 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:39.286711    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d"
Nov 04 02:28:39.354275 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:39.354236    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-maas-api-xks-ca\") pod \"db5f36dd-8c7f-441f-aaae-dd72fb1532ec\" (UID: \"db5f36dd-8c7f-441f-aaae-dd72fb1532ec\") "
Nov 04 02:28:39.354483 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:39.354435    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-maas-api-openshift-ca\") pod \"db5f36dd-8c7f-441f-aaae-dd72fb1532ec\" (UID: \"db5f36dd-8c7f-441f-aaae-dd72fb1532ec\") "
Nov 04 02:28:39.354530 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:39.354479    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-kube-api-access-t97xp\" (UniqueName: \"kubernetes.io/projected/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-kube-api-access-t97xp\") pod \"db5f36dd-8c7f-441f-aaae-dd72fb1532ec\" (UID: \"db5f36dd-8c7f-441f-aaae-dd72fb1532ec\") "
Nov 04 02:28:39.354835 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:39.354806    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-maas-api-openshift-ca" pod "db5f36dd-8c7f-441f-aaae-dd72fb1532ec" (UID: "db5f36dd-8c7f-441f-aaae-dd72fb1532ec"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:28:39.356398 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:39.356376    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-maas-api-xks-ca" pod "db5f36dd-8c7f-441f-aaae-dd72fb1532ec" (UID: "db5f36dd-8c7f-441f-aaae-dd72fb1532ec"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:28:39.356520 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:39.356498    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-kube-api-access-t97xp" pod "db5f36dd-8c7f-441f-aaae-dd72fb1532ec" (UID: "db5f36dd-8c7f-441f-aaae-dd72fb1532ec"). InnerVolumeSpecName "kube-api-access-t97xp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:28:39.455653 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:39.455618    4865 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-maas-api-openshift-ca\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:39.455653 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:39.455648    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t97xp\" (UniqueName: \"kubernetes.io/projected/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-kube-api-access-t97xp\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:39.455653 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:39.455658    4865 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/db5f36dd-8c7f-441f-aaae-dd72fb1532ec-maas-api-xks-ca\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:40.112041 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:40.112006    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:28:40.161313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:40.161266    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d" event={"ID":"db5f36dd-8c7f-441f-aaae-dd72fb1532ec","Type":"ContainerDied","Data":"c6ea92eafa8f875ddc11ebda3dec69a65fcfb0685021e6c4924f4a509732fcff"}
Nov 04 02:28:40.161313 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:40.161304    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01874c89-jkc6d"
Nov 04 02:28:40.161518 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:40.161332    4865 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6ea92eafa8f875ddc11ebda3dec69a65fcfb0685021e6c4924f4a509732fcff"
Nov 04 02:28:57.253837 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:28:57.253802    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gdhhm"]
Nov 04 02:28:58.403183 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:28:58.403147    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:29:18.530649 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:18.530572    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"]
Nov 04 02:29:18.531043 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:18.530780    4865 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7" podUID="addb3654-9be9-49e2-ba5d-9a351d932eb3" containerName="istio-proxy" containerID="cri-o://354a92e997367e9ee849e994827f7a37eec86b1c82a8f8fdb067fd7a40382555" gracePeriod=30
Nov 04 02:29:21.841674 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:21.841644    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-gdhhm_ac4b347b-ef59-44da-a807-8dc79596178e/limitador/0.log"
Nov 04 02:29:23.133215 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.133188    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7656484495-trb67_574b6242-13d5-467e-b940-e67d73036f08/manager/0.log"
Nov 04 02:29:23.782752 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.782724    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:29:23.890735 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.890664    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/addb3654-9be9-49e2-ba5d-9a351d932eb3-kube-api-access-wctx2\" (UniqueName: \"kubernetes.io/projected/addb3654-9be9-49e2-ba5d-9a351d932eb3-kube-api-access-wctx2\") pod \"addb3654-9be9-49e2-ba5d-9a351d932eb3\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") "
Nov 04 02:29:23.890735 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.890710    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/addb3654-9be9-49e2-ba5d-9a351d932eb3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/addb3654-9be9-49e2-ba5d-9a351d932eb3-istiod-ca-cert\") pod \"addb3654-9be9-49e2-ba5d-9a351d932eb3\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") "
Nov 04 02:29:23.890735 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.890736    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-credential-socket\") pod \"addb3654-9be9-49e2-ba5d-9a351d932eb3\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") "
Nov 04 02:29:23.891082 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.890760    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-podinfo\") pod \"addb3654-9be9-49e2-ba5d-9a351d932eb3\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") "
Nov 04 02:29:23.891082 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.890785    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-workload-socket\") pod \"addb3654-9be9-49e2-ba5d-9a351d932eb3\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") "
Nov 04 02:29:23.891082 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.890848    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-token\" (UniqueName: \"kubernetes.io/projected/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-token\") pod \"addb3654-9be9-49e2-ba5d-9a351d932eb3\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") "
Nov 04 02:29:23.891082 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.890885    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-data\") pod \"addb3654-9be9-49e2-ba5d-9a351d932eb3\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") "
Nov 04 02:29:23.891082 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.890915    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-envoy\") pod \"addb3654-9be9-49e2-ba5d-9a351d932eb3\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") "
Nov 04 02:29:23.891082 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.890955    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-workload-certs\") pod \"addb3654-9be9-49e2-ba5d-9a351d932eb3\" (UID: \"addb3654-9be9-49e2-ba5d-9a351d932eb3\") "
Nov 04 02:29:23.891411 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.891076    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-credential-socket" pod "addb3654-9be9-49e2-ba5d-9a351d932eb3" (UID: "addb3654-9be9-49e2-ba5d-9a351d932eb3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:29:23.891411 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.891239    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/addb3654-9be9-49e2-ba5d-9a351d932eb3-istiod-ca-cert" pod "addb3654-9be9-49e2-ba5d-9a351d932eb3" (UID: "addb3654-9be9-49e2-ba5d-9a351d932eb3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:29:23.891411 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.891221    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-workload-socket" pod "addb3654-9be9-49e2-ba5d-9a351d932eb3" (UID: "addb3654-9be9-49e2-ba5d-9a351d932eb3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:29:23.891411 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.891306    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-workload-certs" pod "addb3654-9be9-49e2-ba5d-9a351d932eb3" (UID: "addb3654-9be9-49e2-ba5d-9a351d932eb3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:29:23.891411 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.891329    4865 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-credential-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:23.891411 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.891332    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-data" pod "addb3654-9be9-49e2-ba5d-9a351d932eb3" (UID: "addb3654-9be9-49e2-ba5d-9a351d932eb3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:29:23.893135 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.893106    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-token" pod "addb3654-9be9-49e2-ba5d-9a351d932eb3" (UID: "addb3654-9be9-49e2-ba5d-9a351d932eb3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:29:23.893135 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.893129    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-envoy" pod "addb3654-9be9-49e2-ba5d-9a351d932eb3" (UID: "addb3654-9be9-49e2-ba5d-9a351d932eb3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:29:23.893328 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.893240    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-podinfo" pod "addb3654-9be9-49e2-ba5d-9a351d932eb3" (UID: "addb3654-9be9-49e2-ba5d-9a351d932eb3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:29:23.893328 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.893262    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/addb3654-9be9-49e2-ba5d-9a351d932eb3-kube-api-access-wctx2" pod "addb3654-9be9-49e2-ba5d-9a351d932eb3" (UID: "addb3654-9be9-49e2-ba5d-9a351d932eb3"). InnerVolumeSpecName "kube-api-access-wctx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:29:23.993212 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.992801    4865 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-token\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:23.993212 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.992835    4865 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-data\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:23.993212 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.992850    4865 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-envoy\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:23.993212 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.992864    4865 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-workload-certs\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:23.993212 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.992880    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wctx2\" (UniqueName: \"kubernetes.io/projected/addb3654-9be9-49e2-ba5d-9a351d932eb3-kube-api-access-wctx2\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:23.993212 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.992895    4865 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/addb3654-9be9-49e2-ba5d-9a351d932eb3-istiod-ca-cert\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:23.993212 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.992908    4865 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/addb3654-9be9-49e2-ba5d-9a351d932eb3-istio-podinfo\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:23.993212 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:23.992922    4865 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/addb3654-9be9-49e2-ba5d-9a351d932eb3-workload-socket\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:24.386131 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:24.386099    4865 generic.go:356] "Generic (PLEG): container finished" podID="addb3654-9be9-49e2-ba5d-9a351d932eb3" containerID="354a92e997367e9ee849e994827f7a37eec86b1c82a8f8fdb067fd7a40382555" exitCode=0
Nov 04 02:29:24.386600 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:24.386146    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"
Nov 04 02:29:24.386600 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:24.386151    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7" event={"ID":"addb3654-9be9-49e2-ba5d-9a351d932eb3","Type":"ContainerDied","Data":"354a92e997367e9ee849e994827f7a37eec86b1c82a8f8fdb067fd7a40382555"}
Nov 04 02:29:24.386600 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:24.386195    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7" event={"ID":"addb3654-9be9-49e2-ba5d-9a351d932eb3","Type":"ContainerDied","Data":"d3b9851ba4eb31bb0456eb77ddfe6f8b89e9c22aa9f44d7775a8b608fa201c10"}
Nov 04 02:29:24.386600 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:24.386216    4865 scope.go:122] "RemoveContainer" containerID="354a92e997367e9ee849e994827f7a37eec86b1c82a8f8fdb067fd7a40382555"
Nov 04 02:29:24.396450 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:24.396424    4865 scope.go:122] "RemoveContainer" containerID="b67d41af9c6d901f74667d0b885483d9da47f495e4cb3cf458240b525394399f"
Nov 04 02:29:24.404080 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:24.404054    4865 scope.go:122] "RemoveContainer" containerID="354a92e997367e9ee849e994827f7a37eec86b1c82a8f8fdb067fd7a40382555"
Nov 04 02:29:24.404510 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:29:24.404487    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"354a92e997367e9ee849e994827f7a37eec86b1c82a8f8fdb067fd7a40382555\": container with ID starting with 354a92e997367e9ee849e994827f7a37eec86b1c82a8f8fdb067fd7a40382555 not found: ID does not exist" containerID="354a92e997367e9ee849e994827f7a37eec86b1c82a8f8fdb067fd7a40382555"
Nov 04 02:29:24.404607 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:24.404523    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"354a92e997367e9ee849e994827f7a37eec86b1c82a8f8fdb067fd7a40382555"} err="failed to get container status \"354a92e997367e9ee849e994827f7a37eec86b1c82a8f8fdb067fd7a40382555\": rpc error: code = NotFound desc = could not find container \"354a92e997367e9ee849e994827f7a37eec86b1c82a8f8fdb067fd7a40382555\": container with ID starting with 354a92e997367e9ee849e994827f7a37eec86b1c82a8f8fdb067fd7a40382555 not found: ID does not exist"
Nov 04 02:29:24.404607 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:24.404547    4865 scope.go:122] "RemoveContainer" containerID="b67d41af9c6d901f74667d0b885483d9da47f495e4cb3cf458240b525394399f"
Nov 04 02:29:24.404816 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:29:24.404796    4865 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b67d41af9c6d901f74667d0b885483d9da47f495e4cb3cf458240b525394399f\": container with ID starting with b67d41af9c6d901f74667d0b885483d9da47f495e4cb3cf458240b525394399f not found: ID does not exist" containerID="b67d41af9c6d901f74667d0b885483d9da47f495e4cb3cf458240b525394399f"
Nov 04 02:29:24.404859 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:24.404825    4865 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b67d41af9c6d901f74667d0b885483d9da47f495e4cb3cf458240b525394399f"} err="failed to get container status \"b67d41af9c6d901f74667d0b885483d9da47f495e4cb3cf458240b525394399f\": rpc error: code = NotFound desc = could not find container \"b67d41af9c6d901f74667d0b885483d9da47f495e4cb3cf458240b525394399f\": container with ID starting with b67d41af9c6d901f74667d0b885483d9da47f495e4cb3cf458240b525394399f not found: ID does not exist"
Nov 04 02:29:24.556067 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:24.556029    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"]
Nov 04 02:29:24.606715 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:24.606684    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7"]
Nov 04 02:29:24.708265 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:24.708236    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-l8pbh_e9e9997e-029b-4d43-a61e-c54bed553dfe/manager/0.log"
Nov 04 02:29:24.895934 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:29:24.895900    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="addb3654-9be9-49e2-ba5d-9a351d932eb3" path="/var/lib/kubelet/pods/addb3654-9be9-49e2-ba5d-9a351d932eb3/volumes"
Nov 04 02:29:28.402767 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:29:28.402730    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:29:58.402357 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:29:58.402319    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:30:00.169899 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.169866    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r"]
Nov 04 02:30:00.170427 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.170257    4865 cpu_manager.go:414] "containerMap: removing container" podUID="db5f36dd-8c7f-441f-aaae-dd72fb1532ec" containerName="revoke-keys"
Nov 04 02:30:00.170427 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.170269    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db5f36dd-8c7f-441f-aaae-dd72fb1532ec" containerName="revoke-keys"
Nov 04 02:30:00.170427 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.170307    4865 cpu_manager.go:414] "containerMap: removing container" podUID="addb3654-9be9-49e2-ba5d-9a351d932eb3" containerName="istio-proxy"
Nov 04 02:30:00.170427 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.170312    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="addb3654-9be9-49e2-ba5d-9a351d932eb3" containerName="istio-proxy"
Nov 04 02:30:00.170427 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.170320    4865 cpu_manager.go:414] "containerMap: removing container" podUID="addb3654-9be9-49e2-ba5d-9a351d932eb3" containerName="istio-proxy"
Nov 04 02:30:00.170427 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.170325    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="addb3654-9be9-49e2-ba5d-9a351d932eb3" containerName="istio-proxy"
Nov 04 02:30:00.170427 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.170397    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="db5f36dd-8c7f-441f-aaae-dd72fb1532ec" containerName="revoke-keys"
Nov 04 02:30:00.170427 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.170408    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="addb3654-9be9-49e2-ba5d-9a351d932eb3" containerName="istio-proxy"
Nov 04 02:30:00.170427 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.170413    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="addb3654-9be9-49e2-ba5d-9a351d932eb3" containerName="istio-proxy"
Nov 04 02:30:00.173516 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.173499    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r"
Nov 04 02:30:00.228716 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.228683    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr8bd\" (UniqueName: \"kubernetes.io/projected/0330215e-5fab-4870-9b23-7835e9130f81-kube-api-access-gr8bd\") pod \"maas-api-key-cleanup-29895990-shv5r\" (UID: \"0330215e-5fab-4870-9b23-7835e9130f81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r"
Nov 04 02:30:00.229020 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.228996    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0330215e-5fab-4870-9b23-7835e9130f81-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895990-shv5r\" (UID: \"0330215e-5fab-4870-9b23-7835e9130f81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r"
Nov 04 02:30:00.230879 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.230855    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r"]
Nov 04 02:30:00.278974 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.278937    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 02:30:00.278974 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.278937    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 02:30:00.279184 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.279006    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:30:00.329667 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.329632    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr8bd\" (UniqueName: \"kubernetes.io/projected/0330215e-5fab-4870-9b23-7835e9130f81-kube-api-access-gr8bd\") pod \"maas-api-key-cleanup-29895990-shv5r\" (UID: \"0330215e-5fab-4870-9b23-7835e9130f81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r"
Nov 04 02:30:00.329864 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.329725    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0330215e-5fab-4870-9b23-7835e9130f81-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895990-shv5r\" (UID: \"0330215e-5fab-4870-9b23-7835e9130f81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r"
Nov 04 02:30:00.330409 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.330388    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0330215e-5fab-4870-9b23-7835e9130f81-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895990-shv5r\" (UID: \"0330215e-5fab-4870-9b23-7835e9130f81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r"
Nov 04 02:30:00.391938 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.391906    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr8bd\" (UniqueName: \"kubernetes.io/projected/0330215e-5fab-4870-9b23-7835e9130f81-kube-api-access-gr8bd\") pod \"maas-api-key-cleanup-29895990-shv5r\" (UID: \"0330215e-5fab-4870-9b23-7835e9130f81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r"
Nov 04 02:30:00.486600 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.486562    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r"
Nov 04 02:30:00.856125 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:00.856101    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r"]
Nov 04 02:30:00.857995 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:30:00.857968    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0330215e_5fab_4870_9b23_7835e9130f81.slice/crio-fa6eb1011a74a6bec773c7fdee296099fad6099ccf9bb875acea684120677de6 WatchSource:0}: Error finding container fa6eb1011a74a6bec773c7fdee296099fad6099ccf9bb875acea684120677de6: Status 404 returned error can't find the container with id fa6eb1011a74a6bec773c7fdee296099fad6099ccf9bb875acea684120677de6
Nov 04 02:30:01.574467 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:01.574430    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r" event={"ID":"0330215e-5fab-4870-9b23-7835e9130f81","Type":"ContainerStarted","Data":"7fccc275812997122f16679fd8683b4b1a2a5a87bed6d595d35702ab181f3364"}
Nov 04 02:30:01.574869 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:01.574489    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r" event={"ID":"0330215e-5fab-4870-9b23-7835e9130f81","Type":"ContainerStarted","Data":"fa6eb1011a74a6bec773c7fdee296099fad6099ccf9bb875acea684120677de6"}
Nov 04 02:30:01.692775 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:01.692723    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r" podStartSLOduration=1.692704643 podStartE2EDuration="1.692704643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:30:01.692031175 +0000 UTC m=+2064.918817873" watchObservedRunningTime="2026-11-04 02:30:01.692704643 +0000 UTC m=+2064.919491340"
Nov 04 02:30:01.761848 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:01.761824    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7656484495-trb67_574b6242-13d5-467e-b940-e67d73036f08/manager/0.log"
Nov 04 02:30:02.580204 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:02.580170    4865 generic.go:356] "Generic (PLEG): container finished" podID="0330215e-5fab-4870-9b23-7835e9130f81" containerID="7fccc275812997122f16679fd8683b4b1a2a5a87bed6d595d35702ab181f3364" exitCode=60
Nov 04 02:30:02.580642 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:02.580221    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r" event={"ID":"0330215e-5fab-4870-9b23-7835e9130f81","Type":"ContainerDied","Data":"7fccc275812997122f16679fd8683b4b1a2a5a87bed6d595d35702ab181f3364"}
Nov 04 02:30:02.580642 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:02.580476    4865 scope.go:122] "RemoveContainer" containerID="7fccc275812997122f16679fd8683b4b1a2a5a87bed6d595d35702ab181f3364"
Nov 04 02:30:03.549843 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:03.549755    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-l8pbh_e9e9997e-029b-4d43-a61e-c54bed553dfe/manager/0.log"
Nov 04 02:30:03.586907 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:03.586874    4865 generic.go:356] "Generic (PLEG): container finished" podID="0330215e-5fab-4870-9b23-7835e9130f81" containerID="10856a7b6a158df938b1b28a4495faa7de014132bc9194555d68d22456645e50" exitCode=60
Nov 04 02:30:03.587362 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:03.586960    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r" event={"ID":"0330215e-5fab-4870-9b23-7835e9130f81","Type":"ContainerDied","Data":"10856a7b6a158df938b1b28a4495faa7de014132bc9194555d68d22456645e50"}
Nov 04 02:30:03.587362 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:03.587070    4865 scope.go:122] "RemoveContainer" containerID="7fccc275812997122f16679fd8683b4b1a2a5a87bed6d595d35702ab181f3364"
Nov 04 02:30:03.587362 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:03.587191    4865 scope.go:122] "RemoveContainer" containerID="10856a7b6a158df938b1b28a4495faa7de014132bc9194555d68d22456645e50"
Nov 04 02:30:03.587531 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:30:03.587477    4865 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895990-shv5r_odh-ai-gateway-infra(0330215e-5fab-4870-9b23-7835e9130f81)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r" podUID="0330215e-5fab-4870-9b23-7835e9130f81"
Nov 04 02:30:04.593644 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:04.593616    4865 scope.go:122] "RemoveContainer" containerID="10856a7b6a158df938b1b28a4495faa7de014132bc9194555d68d22456645e50"
Nov 04 02:30:04.594039 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:30:04.593849    4865 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895990-shv5r_odh-ai-gateway-infra(0330215e-5fab-4870-9b23-7835e9130f81)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r" podUID="0330215e-5fab-4870-9b23-7835e9130f81"
Nov 04 02:30:13.888805 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:13.888772    4865 scope.go:122] "RemoveContainer" containerID="10856a7b6a158df938b1b28a4495faa7de014132bc9194555d68d22456645e50"
Nov 04 02:30:14.648458 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:14.648430    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r" event={"ID":"0330215e-5fab-4870-9b23-7835e9130f81","Type":"ContainerStarted","Data":"f47c2654a4e8cdedc54ee5d7598579c32804f88572e59fcc5436a51333564efc"}
Nov 04 02:30:14.648849 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:14.648830    4865 scope.go:122] "RemoveContainer" containerID="f47c2654a4e8cdedc54ee5d7598579c32804f88572e59fcc5436a51333564efc"
Nov 04 02:30:14.649114 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:30:14.649093    4865 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29895990-shv5r_odh-ai-gateway-infra(0330215e-5fab-4870-9b23-7835e9130f81)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r" podUID="0330215e-5fab-4870-9b23-7835e9130f81"
Nov 04 02:30:15.507205 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:15.507173    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29895990-shv5r_0330215e-5fab-4870-9b23-7835e9130f81/cleanup/2.log"
Nov 04 02:30:15.655191 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:15.655161    4865 generic.go:356] "Generic (PLEG): container finished" podID="0330215e-5fab-4870-9b23-7835e9130f81" containerID="f47c2654a4e8cdedc54ee5d7598579c32804f88572e59fcc5436a51333564efc" exitCode=60
Nov 04 02:30:15.655378 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:15.655207    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r" event={"ID":"0330215e-5fab-4870-9b23-7835e9130f81","Type":"ContainerDied","Data":"f47c2654a4e8cdedc54ee5d7598579c32804f88572e59fcc5436a51333564efc"}
Nov 04 02:30:15.655378 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:15.655253    4865 scope.go:122] "RemoveContainer" containerID="10856a7b6a158df938b1b28a4495faa7de014132bc9194555d68d22456645e50"
Nov 04 02:30:15.655585 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:15.655560    4865 scope.go:122] "RemoveContainer" containerID="f47c2654a4e8cdedc54ee5d7598579c32804f88572e59fcc5436a51333564efc"
Nov 04 02:30:15.655808 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:30:15.655787    4865 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29895990-shv5r_odh-ai-gateway-infra(0330215e-5fab-4870-9b23-7835e9130f81)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r" podUID="0330215e-5fab-4870-9b23-7835e9130f81"
Nov 04 02:30:15.745112 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:15.745082    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-01874c89-jkc6d_db5f36dd-8c7f-441f-aaae-dd72fb1532ec/revoke-keys/0.log"
Nov 04 02:30:15.778820 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:15.778754    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r"]
Nov 04 02:30:16.233525 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:16.233499    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-4ff54f84-vzcxr_781ba848-3afb-4372-87dd-70d5279f2f2e/revoke-keys/0.log"
Nov 04 02:30:16.796002 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:16.795975    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r"
Nov 04 02:30:16.888785 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:16.888751    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0330215e-5fab-4870-9b23-7835e9130f81-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0330215e-5fab-4870-9b23-7835e9130f81-maas-api-service-ca\") pod \"0330215e-5fab-4870-9b23-7835e9130f81\" (UID: \"0330215e-5fab-4870-9b23-7835e9130f81\") "
Nov 04 02:30:16.888945 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:16.888901    4865 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0330215e-5fab-4870-9b23-7835e9130f81-kube-api-access-gr8bd\" (UniqueName: \"kubernetes.io/projected/0330215e-5fab-4870-9b23-7835e9130f81-kube-api-access-gr8bd\") pod \"0330215e-5fab-4870-9b23-7835e9130f81\" (UID: \"0330215e-5fab-4870-9b23-7835e9130f81\") "
Nov 04 02:30:16.889093 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:16.889072    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0330215e-5fab-4870-9b23-7835e9130f81-maas-api-service-ca" pod "0330215e-5fab-4870-9b23-7835e9130f81" (UID: "0330215e-5fab-4870-9b23-7835e9130f81"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:30:16.891096 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:16.891074    4865 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0330215e-5fab-4870-9b23-7835e9130f81-kube-api-access-gr8bd" pod "0330215e-5fab-4870-9b23-7835e9130f81" (UID: "0330215e-5fab-4870-9b23-7835e9130f81"). InnerVolumeSpecName "kube-api-access-gr8bd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:30:16.990491 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:16.990456    4865 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0330215e-5fab-4870-9b23-7835e9130f81-maas-api-service-ca\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:30:16.990491 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:16.990482    4865 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gr8bd\" (UniqueName: \"kubernetes.io/projected/0330215e-5fab-4870-9b23-7835e9130f81-kube-api-access-gr8bd\") on node \"ip-10-0-134-204.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:30:17.667088 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:17.667059    4865 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r"
Nov 04 02:30:17.667251 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:17.667056    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r" event={"ID":"0330215e-5fab-4870-9b23-7835e9130f81","Type":"ContainerDied","Data":"fa6eb1011a74a6bec773c7fdee296099fad6099ccf9bb875acea684120677de6"}
Nov 04 02:30:17.667251 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:17.667223    4865 scope.go:122] "RemoveContainer" containerID="f47c2654a4e8cdedc54ee5d7598579c32804f88572e59fcc5436a51333564efc"
Nov 04 02:30:17.834075 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:17.834036    4865 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r"]
Nov 04 02:30:17.837939 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:17.837915    4865 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895990-shv5r"]
Nov 04 02:30:18.899144 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:18.899102    4865 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0330215e-5fab-4870-9b23-7835e9130f81" path="/var/lib/kubelet/pods/0330215e-5fab-4870-9b23-7835e9130f81/volumes"
Nov 04 02:30:20.607789 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:20.607748    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-gdhhm_ac4b347b-ef59-44da-a807-8dc79596178e/limitador/0.log"
Nov 04 02:30:22.122132 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:22.122101    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-8pm7p_4276cf3a-e09a-436a-9e38-a87ba1e80f30/discovery/0.log"
Nov 04 02:30:28.403416 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:30:28.403377    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:30:32.572829 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:32.572793    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77fbbd9bf8-td596_0e12e95d-d3af-4ec4-821a-e580ec557555/kube-auth-proxy/0.log"
Nov 04 02:30:32.811603 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:32.811493    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p_aec82805-4138-4238-8b24-44baa8dc2ba4/istio-proxy/0.log"
Nov 04 02:30:33.332445 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:33.332400    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-b9chv_466445d6-ee70-4113-99af-6a9b562b5659/istio-proxy/0.log"
Nov 04 02:30:33.562106 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:33.562077    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd_ffac90da-23d5-4cf1-9dcf-f543bc11736b/istio-proxy/0.log"
Nov 04 02:30:34.020525 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:34.020497    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw_15993036-6c89-4844-b870-ccb12cfc744a/istio-proxy/0.log"
Nov 04 02:30:35.440881 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:35.440852    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-7hkvk_0032af18-ab69-46ad-b078-d61f927b22ca/payload-pre-processing/0.log"
Nov 04 02:30:35.668316 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:35.668247    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-ckdnq_70290b6d-1703-43ad-a7e5-42c79eed7e1a/payload-processing/0.log"
Nov 04 02:30:35.894170 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:35.894144    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56bdbb678c-mvkmq_cc72c8e1-7e7c-4f53-946b-6ef94d6c5971/router/0.log"
Nov 04 02:30:37.115517 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:37.115484    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/2.log"
Nov 04 02:30:37.123535 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:37.123511    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jlqmg_417744b4-5ed7-4b20-9960-7147d2d567ee/console-operator/2.log"
Nov 04 02:30:37.507395 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:37.507360    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-c7flb_54832357-9824-4b08-b0a6-f58dcc2f768b/main/0.log"
Nov 04 02:30:37.982906 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:37.982875    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-9cbzm_2c1f5846-cfc2-4b97-b24e-c2b99105ef98/main/0.log"
Nov 04 02:30:38.214694 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:38.214660    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl_15039fbe-55b4-4b82-9b2d-78d24b0ed5b3/main/0.log"
Nov 04 02:30:38.924528 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:38.924494    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp_f7dc3968-3684-408e-9d43-568adb008d30/main/0.log"
Nov 04 02:30:49.464176 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.464100    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-w2sfr/must-gather-2vvwm"]
Nov 04 02:30:49.464600 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.464561    4865 cpu_manager.go:414] "containerMap: removing container" podUID="0330215e-5fab-4870-9b23-7835e9130f81" containerName="cleanup"
Nov 04 02:30:49.464600 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.464577    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0330215e-5fab-4870-9b23-7835e9130f81" containerName="cleanup"
Nov 04 02:30:49.464721 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.464605    4865 cpu_manager.go:414] "containerMap: removing container" podUID="0330215e-5fab-4870-9b23-7835e9130f81" containerName="cleanup"
Nov 04 02:30:49.464721 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.464613    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0330215e-5fab-4870-9b23-7835e9130f81" containerName="cleanup"
Nov 04 02:30:49.464825 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.464742    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="0330215e-5fab-4870-9b23-7835e9130f81" containerName="cleanup"
Nov 04 02:30:49.464825 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.464758    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="0330215e-5fab-4870-9b23-7835e9130f81" containerName="cleanup"
Nov 04 02:30:49.464894 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.464855    4865 cpu_manager.go:414] "containerMap: removing container" podUID="0330215e-5fab-4870-9b23-7835e9130f81" containerName="cleanup"
Nov 04 02:30:49.464894 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.464862    4865 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0330215e-5fab-4870-9b23-7835e9130f81" containerName="cleanup"
Nov 04 02:30:49.464955 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.464934    4865 memory_manager.go:365] "RemoveStaleState removing state" podUID="0330215e-5fab-4870-9b23-7835e9130f81" containerName="cleanup"
Nov 04 02:30:49.469110 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.469092    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w2sfr/must-gather-2vvwm"
Nov 04 02:30:49.497885 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.497858    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1295fc8f-ab07-4796-b0e6-e4dabde375a4-must-gather-output\") pod \"must-gather-2vvwm\" (UID: \"1295fc8f-ab07-4796-b0e6-e4dabde375a4\") " pod="openshift-must-gather-w2sfr/must-gather-2vvwm"
Nov 04 02:30:49.498005 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.497899    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfg4f\" (UniqueName: \"kubernetes.io/projected/1295fc8f-ab07-4796-b0e6-e4dabde375a4-kube-api-access-pfg4f\") pod \"must-gather-2vvwm\" (UID: \"1295fc8f-ab07-4796-b0e6-e4dabde375a4\") " pod="openshift-must-gather-w2sfr/must-gather-2vvwm"
Nov 04 02:30:49.525147 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.525100    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w2sfr\"/\"kube-root-ca.crt\""
Nov 04 02:30:49.575920 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.575881    4865 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w2sfr\"/\"openshift-service-ca.crt\""
Nov 04 02:30:49.576124 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.576005    4865 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-w2sfr\"/\"default-dockercfg-lvbsf\""
Nov 04 02:30:49.577814 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.577792    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w2sfr/must-gather-2vvwm"]
Nov 04 02:30:49.598697 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.598662    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1295fc8f-ab07-4796-b0e6-e4dabde375a4-must-gather-output\") pod \"must-gather-2vvwm\" (UID: \"1295fc8f-ab07-4796-b0e6-e4dabde375a4\") " pod="openshift-must-gather-w2sfr/must-gather-2vvwm"
Nov 04 02:30:49.598845 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.598718    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfg4f\" (UniqueName: \"kubernetes.io/projected/1295fc8f-ab07-4796-b0e6-e4dabde375a4-kube-api-access-pfg4f\") pod \"must-gather-2vvwm\" (UID: \"1295fc8f-ab07-4796-b0e6-e4dabde375a4\") " pod="openshift-must-gather-w2sfr/must-gather-2vvwm"
Nov 04 02:30:49.599003 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.598984    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1295fc8f-ab07-4796-b0e6-e4dabde375a4-must-gather-output\") pod \"must-gather-2vvwm\" (UID: \"1295fc8f-ab07-4796-b0e6-e4dabde375a4\") " pod="openshift-must-gather-w2sfr/must-gather-2vvwm"
Nov 04 02:30:49.661216 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.661182    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfg4f\" (UniqueName: \"kubernetes.io/projected/1295fc8f-ab07-4796-b0e6-e4dabde375a4-kube-api-access-pfg4f\") pod \"must-gather-2vvwm\" (UID: \"1295fc8f-ab07-4796-b0e6-e4dabde375a4\") " pod="openshift-must-gather-w2sfr/must-gather-2vvwm"
Nov 04 02:30:49.782310 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:49.782209    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w2sfr/must-gather-2vvwm"
Nov 04 02:30:50.155379 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:50.155354    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w2sfr/must-gather-2vvwm"]
Nov 04 02:30:50.156947 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:30:50.156915    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1295fc8f_ab07_4796_b0e6_e4dabde375a4.slice/crio-8a6708b9418e5409a1c7d0dd2d6b56bf62d38d24c39b3bf5cead75006659189d WatchSource:0}: Error finding container 8a6708b9418e5409a1c7d0dd2d6b56bf62d38d24c39b3bf5cead75006659189d: Status 404 returned error can't find the container with id 8a6708b9418e5409a1c7d0dd2d6b56bf62d38d24c39b3bf5cead75006659189d
Nov 04 02:30:50.158901 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:50.158885    4865 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:30:50.825359 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:50.825312    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w2sfr/must-gather-2vvwm" event={"ID":"1295fc8f-ab07-4796-b0e6-e4dabde375a4","Type":"ContainerStarted","Data":"8a6708b9418e5409a1c7d0dd2d6b56bf62d38d24c39b3bf5cead75006659189d"}
Nov 04 02:30:51.831936 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:51.831896    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w2sfr/must-gather-2vvwm" event={"ID":"1295fc8f-ab07-4796-b0e6-e4dabde375a4","Type":"ContainerStarted","Data":"86c7cb92f6fd218ab0d389dcdf02707fccc610248eb58e67107610eac3a92cbd"}
Nov 04 02:30:51.831936 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:51.831940    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w2sfr/must-gather-2vvwm" event={"ID":"1295fc8f-ab07-4796-b0e6-e4dabde375a4","Type":"ContainerStarted","Data":"0d71a3b290b47b4ca989ad8138b2df37b70f2f379ccd3b1a0d2171c9312e00e8"}
Nov 04 02:30:51.951076 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:51.951020    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-w2sfr/must-gather-2vvwm" podStartSLOduration=1.6893704939999998 podStartE2EDuration="2.951006196s" totalImagesPullingTime="1.261635702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:30:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:30:51.948902918 +0000 UTC m=+2115.175689587" watchObservedRunningTime="2026-11-04 02:30:51.951006196 +0000 UTC m=+2115.177792920"
Nov 04 02:30:58.492709 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:30:58.492661    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:30:59.689983 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:30:59.689956    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-x48xl_a994b38f-0096-4fff-b49c-e6d99dc5a6be/global-pull-secret-syncer/0.log"
Nov 04 02:31:00.311304 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:00.310919    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-7gmmg_99ef12a9-a676-4258-a3a8-e7fcd611d158/connection-checker/0.log"
Nov 04 02:31:00.752030 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:00.749904    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-p48wv_ef75aae9-bb5f-4e09-b374-fa398aadfae9/connection-checker/0.log"
Nov 04 02:31:01.183355 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:01.182858    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-rzbgn_c07ce523-d075-4203-be44-3f7d5af7e0d7/connection-checker/0.log"
Nov 04 02:31:02.513340 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:02.511078    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dszxl_b1926369-9621-4cac-b797-43a306bdb709/konnectivity-agent/0.log"
Nov 04 02:31:03.341583 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:03.341553    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-204.us-west-1.compute.internal_56dfafb26cd80e2f2f640fddc6c0403a/haproxy/0.log"
Nov 04 02:31:08.758692 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:08.758657    4865 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"]
Nov 04 02:31:08.769154 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:08.769115    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:08.821276 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:08.821235    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"]
Nov 04 02:31:08.912377 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:08.912341    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f18ff186-3f25-4799-a8c4-052b7c40c2d0-sys\") pod \"perf-node-gather-daemonset-jkwkh\" (UID: \"f18ff186-3f25-4799-a8c4-052b7c40c2d0\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:08.912377 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:08.912386    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f18ff186-3f25-4799-a8c4-052b7c40c2d0-proc\") pod \"perf-node-gather-daemonset-jkwkh\" (UID: \"f18ff186-3f25-4799-a8c4-052b7c40c2d0\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:08.912602 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:08.912524    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f18ff186-3f25-4799-a8c4-052b7c40c2d0-podres\") pod \"perf-node-gather-daemonset-jkwkh\" (UID: \"f18ff186-3f25-4799-a8c4-052b7c40c2d0\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:08.912602 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:08.912577    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f18ff186-3f25-4799-a8c4-052b7c40c2d0-lib-modules\") pod \"perf-node-gather-daemonset-jkwkh\" (UID: \"f18ff186-3f25-4799-a8c4-052b7c40c2d0\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:08.912690 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:08.912611    4865 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwqbz\" (UniqueName: \"kubernetes.io/projected/f18ff186-3f25-4799-a8c4-052b7c40c2d0-kube-api-access-zwqbz\") pod \"perf-node-gather-daemonset-jkwkh\" (UID: \"f18ff186-3f25-4799-a8c4-052b7c40c2d0\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:09.013534 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:09.013450    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwqbz\" (UniqueName: \"kubernetes.io/projected/f18ff186-3f25-4799-a8c4-052b7c40c2d0-kube-api-access-zwqbz\") pod \"perf-node-gather-daemonset-jkwkh\" (UID: \"f18ff186-3f25-4799-a8c4-052b7c40c2d0\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:09.013762 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:09.013738    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f18ff186-3f25-4799-a8c4-052b7c40c2d0-sys\") pod \"perf-node-gather-daemonset-jkwkh\" (UID: \"f18ff186-3f25-4799-a8c4-052b7c40c2d0\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:09.013836 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:09.013793    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f18ff186-3f25-4799-a8c4-052b7c40c2d0-proc\") pod \"perf-node-gather-daemonset-jkwkh\" (UID: \"f18ff186-3f25-4799-a8c4-052b7c40c2d0\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:09.013898 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:09.013853    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f18ff186-3f25-4799-a8c4-052b7c40c2d0-podres\") pod \"perf-node-gather-daemonset-jkwkh\" (UID: \"f18ff186-3f25-4799-a8c4-052b7c40c2d0\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:09.013898 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:09.013860    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f18ff186-3f25-4799-a8c4-052b7c40c2d0-sys\") pod \"perf-node-gather-daemonset-jkwkh\" (UID: \"f18ff186-3f25-4799-a8c4-052b7c40c2d0\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:09.013898 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:09.013881    4865 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f18ff186-3f25-4799-a8c4-052b7c40c2d0-lib-modules\") pod \"perf-node-gather-daemonset-jkwkh\" (UID: \"f18ff186-3f25-4799-a8c4-052b7c40c2d0\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:09.014031 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:09.013918    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f18ff186-3f25-4799-a8c4-052b7c40c2d0-proc\") pod \"perf-node-gather-daemonset-jkwkh\" (UID: \"f18ff186-3f25-4799-a8c4-052b7c40c2d0\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:09.014031 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:09.013938    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f18ff186-3f25-4799-a8c4-052b7c40c2d0-lib-modules\") pod \"perf-node-gather-daemonset-jkwkh\" (UID: \"f18ff186-3f25-4799-a8c4-052b7c40c2d0\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:09.014031 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:09.013938    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f18ff186-3f25-4799-a8c4-052b7c40c2d0-podres\") pod \"perf-node-gather-daemonset-jkwkh\" (UID: \"f18ff186-3f25-4799-a8c4-052b7c40c2d0\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:09.080949 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:09.080917    4865 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwqbz\" (UniqueName: \"kubernetes.io/projected/f18ff186-3f25-4799-a8c4-052b7c40c2d0-kube-api-access-zwqbz\") pod \"perf-node-gather-daemonset-jkwkh\" (UID: \"f18ff186-3f25-4799-a8c4-052b7c40c2d0\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:09.092341 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:09.092318    4865 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:09.464968 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:09.464936    4865 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"]
Nov 04 02:31:09.466492 ip-10-0-134-204 kubenswrapper[4865]: W1104 02:31:09.466462    4865 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf18ff186_3f25_4799_a8c4_052b7c40c2d0.slice/crio-e0b94d742a7c2cf4fba96c4e5087f8cdcd9a5340d1c4897d8f8218309b67cac0 WatchSource:0}: Error finding container e0b94d742a7c2cf4fba96c4e5087f8cdcd9a5340d1c4897d8f8218309b67cac0: Status 404 returned error can't find the container with id e0b94d742a7c2cf4fba96c4e5087f8cdcd9a5340d1c4897d8f8218309b67cac0
Nov 04 02:31:10.014667 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:10.014618    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh" event={"ID":"f18ff186-3f25-4799-a8c4-052b7c40c2d0","Type":"ContainerStarted","Data":"5cd6d8a970317c94470ce0cbd3b564dff861f6eb4e489c8a2b2062f545fe5113"}
Nov 04 02:31:10.015145 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:10.014724    4865 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh" event={"ID":"f18ff186-3f25-4799-a8c4-052b7c40c2d0","Type":"ContainerStarted","Data":"e0b94d742a7c2cf4fba96c4e5087f8cdcd9a5340d1c4897d8f8218309b67cac0"}
Nov 04 02:31:10.015145 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:10.014874    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:10.132007 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:10.131945    4865 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh" podStartSLOduration=2.131931367 podStartE2EDuration="2.131931367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:31:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:31:10.131597818 +0000 UTC m=+2133.358384486" watchObservedRunningTime="2026-11-04 02:31:10.131931367 +0000 UTC m=+2133.358718033"
Nov 04 02:31:10.992638 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:10.992607    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-gdhhm_ac4b347b-ef59-44da-a807-8dc79596178e/limitador/0.log"
Nov 04 02:31:11.878667 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:11.878610    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7656484495-trb67_574b6242-13d5-467e-b940-e67d73036f08/manager/0.log"
Nov 04 02:31:13.627729 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:13.627676    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-l8pbh_e9e9997e-029b-4d43-a61e-c54bed553dfe/manager/0.log"
Nov 04 02:31:16.030148 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:16.030113    4865 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-jkwkh"
Nov 04 02:31:16.492822 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:16.492792    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7656484495-trb67_574b6242-13d5-467e-b940-e67d73036f08/manager/0.log"
Nov 04 02:31:17.015175 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:17.015142    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-wzmdz_40363d65-7ece-4f05-8668-439a60582a18/cluster-monitoring-operator/0.log"
Nov 04 02:31:18.280678 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:18.280641    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-l8pbh_e9e9997e-029b-4d43-a61e-c54bed553dfe/manager/0.log"
Nov 04 02:31:21.158995 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:21.158958    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7656484495-trb67_574b6242-13d5-467e-b940-e67d73036f08/manager/0.log"
Nov 04 02:31:22.076066 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:22.076039    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h7xrq_67c30d6f-2598-4efe-9613-37b12c8f0be2/node-exporter/0.log"
Nov 04 02:31:22.498878 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:22.498845    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h7xrq_67c30d6f-2598-4efe-9613-37b12c8f0be2/kube-rbac-proxy/0.log"
Nov 04 02:31:22.919656 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:22.919572    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h7xrq_67c30d6f-2598-4efe-9613-37b12c8f0be2/init-textfile/0.log"
Nov 04 02:31:22.957021 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:22.956984    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-l8pbh_e9e9997e-029b-4d43-a61e-c54bed553dfe/manager/0.log"
Nov 04 02:31:27.421636 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:27.421601    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-8pm7p_4276cf3a-e09a-436a-9e38-a87ba1e80f30/discovery/0.log"
Nov 04 02:31:27.740722 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:27.740690    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77fbbd9bf8-td596_0e12e95d-d3af-4ec4-821a-e580ec557555/kube-auth-proxy/0.log"
Nov 04 02:31:27.955563 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:27.955527    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p_aec82805-4138-4238-8b24-44baa8dc2ba4/istio-proxy/0.log"
Nov 04 02:31:28.381770 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:28.381728    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-b9chv_466445d6-ee70-4113-99af-6a9b562b5659/istio-proxy/0.log"
Nov 04 02:31:28.405724 ip-10-0-134-204 kubenswrapper[4865]: E1104 02:31:28.405683    4865 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fkhvv" containerName="extractor"
Nov 04 02:31:28.600596 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:28.600570    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd_ffac90da-23d5-4cf1-9dcf-f543bc11736b/istio-proxy/0.log"
Nov 04 02:31:28.877068 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:28.877038    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b55bdf9f-dfr2j_aa8a869d-eb02-4634-ba03-3da321d7d0ba/telemeter-client/0.log"
Nov 04 02:31:29.029035 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:29.029006    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw_15993036-6c89-4844-b870-ccb12cfc744a/istio-proxy/0.log"
Nov 04 02:31:29.298535 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:29.298507    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b55bdf9f-dfr2j_aa8a869d-eb02-4634-ba03-3da321d7d0ba/reload/0.log"
Nov 04 02:31:29.718397 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:29.718359    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b55bdf9f-dfr2j_aa8a869d-eb02-4634-ba03-3da321d7d0ba/kube-rbac-proxy/0.log"
Nov 04 02:31:30.207419 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:30.207391    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-7hkvk_0032af18-ab69-46ad-b078-d61f927b22ca/payload-pre-processing/0.log"
Nov 04 02:31:30.419195 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:30.419169    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-ckdnq_70290b6d-1703-43ad-a7e5-42c79eed7e1a/payload-processing/0.log"
Nov 04 02:31:30.635754 ip-10-0-134-204 kubenswrapper[4865]: I1104 02:31:30.635679    4865 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56bdbb678c-mvkmq_cc72c8e1-7e7c-4f53-946b-6ef94d6c5971/router/0.log"