Nov 04 03:16:31.638449 ip-10-0-135-242 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 03:16:32.107838 ip-10-0-135-242 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:16:32.107838 ip-10-0-135-242 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 03:16:32.107838 ip-10-0-135-242 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:16:32.107838 ip-10-0-135-242 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:16:32.112048 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.111971    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:32.112048 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112049    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112058    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112062    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112066    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112069    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112071    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112074    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112076    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112079    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112082    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112084    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112087    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112089    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112092    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112095    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112097    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112100    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112103    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112105    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:32.112114 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112108    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112111    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112113    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112116    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112119    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112123    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112126    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112128    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112132    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112137    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112140    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112143    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112145    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112148    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112151    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112153    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112156    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112174    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112178    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112183    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:32.112568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112187    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112191    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112194    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112196    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112199    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112201    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112206    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112210    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112213    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112216    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112219    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112222    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112224    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112227    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112230    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112233    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112237    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112240    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112242    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112245    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:32.113039 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112247    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112250    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112252    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112255    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112257    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112261    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112264    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112266    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112269    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112271    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112274    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112277    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112279    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112282    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112284    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112287    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112290    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112293    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112295    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112298    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:32.113543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112300    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112303    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112308    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112311    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112314    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112316    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112319    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112322    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112325    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112327    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112330    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112333    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112335    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112338    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112340    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112343    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112345    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112348    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112351    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112353    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:32.114072 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112356    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112359    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112361    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112364    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112366    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.112369    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113144    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113151    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113156    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113174    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113179    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113182    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113186    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113189    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113192    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113195    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113197    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113200    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113204    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:32.114567 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113207    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113210    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113212    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113215    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113218    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113220    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113223    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113226    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113228    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113231    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113234    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113236    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113240    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113243    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113246    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113248    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113251    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113254    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113256    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:32.115056 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113259    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113261    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113264    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113266    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113269    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113271    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113274    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113276    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113279    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113281    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113284    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113287    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113289    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113292    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113294    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113297    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113300    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113302    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113305    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:32.115516 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113307    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113310    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113312    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113315    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113317    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113320    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113322    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113326    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113329    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113331    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113334    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113337    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113340    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113342    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113345    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113348    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113351    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113353    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113355    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:32.115965 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113359    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113363    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113366    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113369    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113371    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113374    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113376    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113379    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113381    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113384    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113387    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113389    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113392    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113394    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113397    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113399    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113402    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113404    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113407    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113413    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:32.116461 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113416    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113418    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113421    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113424    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113427    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113430    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113432    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113435    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113438    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113440    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113443    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113446    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113448    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113451    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113453    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.113456    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114804    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114812    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114816    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114819    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:32.116929 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114822    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114825    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114828    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114830    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114833    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114836    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114838    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114841    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114843    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114846    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114849    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114853    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114856    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114860    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114863    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114866    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114868    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114871    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114873    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114876    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:32.117425 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114878    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114881    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114883    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114886    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114889    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114891    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114894    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114896    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114899    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114902    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114904    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114907    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114909    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114912    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114915    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114917    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114920    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114922    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114925    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114928    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:32.117895 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114931    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114933    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114936    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114940    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114943    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114945    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114950    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114953    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114956    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114960    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114963    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114965    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114968    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114970    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114973    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114975    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114978    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114980    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114983    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:32.118384 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114985    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114988    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114991    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114993    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114996    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.114999    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115001    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115003    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115006    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115008    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115011    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115014    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115016    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115019    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115021    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115024    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115028    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115031    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115034    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115036    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:32.118826 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115040    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115043    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115046    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115048    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115051    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115053    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115056    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115059    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115062    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115064    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115068    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115073    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115078    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115081    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115083    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115086    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115089    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115091    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115096    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:32.119312 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115099    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115102    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115106    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.115109    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116131    2575 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116140    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116147    2575 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116151    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116156    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116172    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116179    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116186    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116190    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116193    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116198    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116202    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116205    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116208    2575 flags.go:64] FLAG: --cgroup-root=""
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116211    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116214    2575 flags.go:64] FLAG: --client-ca-file=""
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116217    2575 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116220    2575 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116225    2575 flags.go:64] FLAG: --cluster-domain=""
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116227    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 03:16:32.119751 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116231    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116234    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116237    2575 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116241    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116244    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116248    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116252    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116255    2575 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116260    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116263    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116266    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116269    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116274    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116277    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116280    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116283    2575 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116286    2575 flags.go:64] FLAG: --enable-server="true"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116289    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116294    2575 flags.go:64] FLAG: --event-burst="100"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116297    2575 flags.go:64] FLAG: --event-qps="50"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116300    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116303    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116306    2575 flags.go:64] FLAG: --eviction-hard=""
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116312    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 03:16:32.120314 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116315    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116318    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116322    2575 flags.go:64] FLAG: --eviction-soft=""
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116325    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116328    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116331    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116334    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116337    2575 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116340    2575 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116343    2575 flags.go:64] FLAG: --feature-gates=""
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116347    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116350    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116354    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116357    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116360    2575 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116363    2575 flags.go:64] FLAG: --help="false"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116366    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116370    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116373    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116376    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116379    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116383    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116386    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 03:16:32.120890 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116389    2575 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116392    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116395    2575 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116400    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116403    2575 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116406    2575 flags.go:64] FLAG: --kube-reserved=""
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116409    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116412    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116415    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116418    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116421    2575 flags.go:64] FLAG: --lock-file=""
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116424    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116427    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116430    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116436    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116439    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116442    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116445    2575 flags.go:64] FLAG: --logging-format="text"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116448    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116451    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116454    2575 flags.go:64] FLAG: --manifest-url=""
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116457    2575 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116461    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116464    2575 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116468    2575 flags.go:64] FLAG: --max-pods="110"
Nov 04 03:16:32.121492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116472    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116475    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116478    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116481    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116484    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116487    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116490    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116497    2575 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116500    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116503    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116507    2575 flags.go:64] FLAG: --pod-cidr=""
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116511    2575 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116514    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116517    2575 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116519    2575 flags.go:64] FLAG: --port="10250"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116523    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116525    2575 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-04a705e1529bc9032"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116529    2575 flags.go:64] FLAG: --qos-reserved=""
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116532    2575 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116535    2575 flags.go:64] FLAG: --register-node="true"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116539    2575 flags.go:64] FLAG: --register-with-taints=""
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116543    2575 flags.go:64] FLAG: --registry-burst="10"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116547    2575 flags.go:64] FLAG: --registry-qps="5"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116550    2575 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116553    2575 flags.go:64] FLAG: --reserved-memory=""
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116557    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 03:16:32.122083 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116560    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116564    2575 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116567    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116570    2575 flags.go:64] FLAG: --runonce="false"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116573    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116577    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116580    2575 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116582    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116585    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116588    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116592    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116595    2575 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116598    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116601    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116604    2575 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116607    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116613    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116618    2575 flags.go:64] FLAG: --system-cgroups=""
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116629    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116643    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116646    2575 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116649    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116654    2575 flags.go:64] FLAG: --tls-min-version=""
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116657    2575 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116660    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 03:16:32.122704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116663    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116666    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116669    2575 flags.go:64] FLAG: --v="2"
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116676    2575 flags.go:64] FLAG: --version="false"
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116680    2575 flags.go:64] FLAG: --vmodule=""
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116684    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.116687    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116847    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116851    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116855    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116859    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116864    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116867    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116870    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116872    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116875    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116878    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116880    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116883    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116885    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116888    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:32.123322 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116890    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116892    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116896    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116899    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116901    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116904    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116907    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116910    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116912    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116915    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116918    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116921    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116924    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116926    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116931    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116934    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116937    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116940    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116943    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116945    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:32.124020 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116948    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116951    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116953    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116956    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116958    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116961    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116963    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116966    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116968    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116971    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116974    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116976    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116979    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116982    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116985    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116987    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116990    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116993    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116995    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.116998    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:32.124684 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117000    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117003    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117006    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117008    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117011    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117014    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117018    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117021    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117023    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117026    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117030    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117034    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117036    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117039    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117042    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117044    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117047    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117050    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117052    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:32.125172 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117055    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117057    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117060    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117063    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117065    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117068    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117070    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117073    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117075    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117078    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117080    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117083    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117085    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117088    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117091    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117093    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117096    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117098    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117101    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117105    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:32.125666 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117108    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:32.126153 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117110    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:32.126153 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117113    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:32.126153 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117116    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:32.126153 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117118    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:32.126153 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117121    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:32.126153 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117123    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:32.126153 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117126    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:32.126153 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117129    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:32.126153 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117131    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:32.126153 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117134    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:32.126153 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117136    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:32.126153 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117139    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:32.126153 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.117177    2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:16:32.126153 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117214    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:32.126153 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117218    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:32.126153 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117222    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117225    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117228    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117230    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117233    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117236    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117238    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117241    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117244    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117246    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117249    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117251    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117254    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117256    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117258    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117262    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117265    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117268    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117271    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:32.126543 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117274    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117277    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117279    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117282    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117284    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117287    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117289    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117292    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117294    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117297    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117299    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117302    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117305    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117307    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117309    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117312    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117315    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117318    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117320    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117323    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:32.126990 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117325    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117328    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117332    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117336    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117339    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117341    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117344    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117347    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117349    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117353    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117356    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117358    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117361    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117363    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117366    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117368    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117371    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117373    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117378    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:32.127481 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117380    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117383    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117386    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117388    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117391    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117394    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117396    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117399    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117401    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117404    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117407    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117409    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117411    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117414    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117416    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117419    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117421    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117424    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117426    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117429    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:32.127940 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117432    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117434    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117439    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117442    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117444    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117447    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117449    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117452    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117454    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117457    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117459    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117463    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117466    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117468    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117471    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117473    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117476    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117478    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117481    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117483    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:32.128410 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117486    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117489    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117491    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117494    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117498    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.117502    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.125890    2575 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.125907    2575 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.125921    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.125926    2575 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.125986    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.125991    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.125994    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.125997    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126000    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126002    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126005    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126008    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126011    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:32.128888 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126013    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126016    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126019    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126022    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126025    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126027    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126030    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126034    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126038    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126041    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126043    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126046    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126049    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126051    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126054    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126056    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126059    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126062    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126064    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126067    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:32.129357 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126070    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126073    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126076    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126078    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126081    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126084    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126086    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126089    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126092    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126094    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126097    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126099    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126102    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126105    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126107    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126110    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126112    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126115    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126118    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126121    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:32.129829 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126124    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126127    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126129    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126132    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126135    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126137    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126140    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126142    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126145    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126147    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126150    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126152    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126155    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126158    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126173    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126177    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126179    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126182    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126184    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:32.130306 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126187    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126190    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126192    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126195    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126197    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126200    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126202    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126206    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126211    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126214    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126218    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126225    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126229    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126231    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126234    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126237    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126240    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126244    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126247    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:32.130773 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126250    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126253    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126255    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126258    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126261    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126263    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126266    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126269    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126272    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126274    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126277    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126280    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126283    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126285    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126288    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126291    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126293    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126296    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:32.131327 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126299    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:32.131764 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.126327    2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:16:32.131764 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126468    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:32.131764 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126472    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:32.131764 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126475    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:32.131764 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126477    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:32.131764 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126480    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:32.131764 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126483    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:32.131764 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126486    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:32.131764 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126489    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:32.131764 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126491    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:32.131764 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126494    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:32.131764 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126497    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:32.131764 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126500    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:32.131764 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126502    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:32.131764 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126505    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:32.131764 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126508    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126511    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126513    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126516    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126518    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126521    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126524    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126526    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126529    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126532    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126534    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126537    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126539    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126542    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126545    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126547    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126550    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126553    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126556    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:32.132255 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126558    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126561    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126564    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126566    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126569    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126572    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126575    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126577    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126580    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126582    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126585    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126587    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126590    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126592    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126595    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126597    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126600    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126604    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126607    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126610    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:32.132703 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126613    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126616    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126619    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126621    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126623    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126626    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126628    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126631    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126634    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126636    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126639    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126641    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126644    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126646    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126649    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126651    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126655    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126659    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126662    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126666    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:32.133283 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126668    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126672    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126674    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126677    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126679    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126682    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126684    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126687    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126690    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126692    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126695    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126698    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126700    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126703    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126705    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126708    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126710    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126713    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126715    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:32.133799 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126718    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:32.134568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126720    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:32.134568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126723    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:32.134568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126725    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:32.134568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126728    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:32.134568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126730    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:32.134568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126733    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:32.134568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126736    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:32.134568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126738    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:32.134568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126741    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:32.134568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126743    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:32.134568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126746    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:32.134568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126752    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:32.134568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126755    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:32.134568 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.126778    2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:16:32.134568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126810    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:32.134568 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126814    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126817    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126820    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126825    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126828    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126831    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126834    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126836    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126839    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126842    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126844    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126847    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126849    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126852    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126854    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126857    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126859    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126862    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126864    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126867    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:32.135094 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126869    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126873    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126877    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126880    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126882    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126885    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126888    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126891    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126893    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126897    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126899    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126902    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126905    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126907    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126910    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126912    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126915    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126917    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126920    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:32.135575 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126922    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126925    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126928    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126930    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126932    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126935    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126938    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126940    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126943    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126946    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126948    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126951    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126953    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126956    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126958    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126961    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126963    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126966    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126968    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126971    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:32.136092 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126973    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126976    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126978    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126982    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126984    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126987    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126989    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126992    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126994    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.126997    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127000    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127003    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127006    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127008    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127011    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127013    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127016    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127018    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127021    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:32.136647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127023    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127026    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127028    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127031    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127033    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127036    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127038    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127041    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127043    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127046    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127048    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127051    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127053    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127056    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127058    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127061    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127063    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127066    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127069    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127072    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:32.137258 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127074    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:32.137793 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127077    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:32.137793 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127080    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:32.137793 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127083    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:32.137793 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127085    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:32.137793 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127088    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:32.137793 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:32.127091    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:32.137793 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.127148    2575 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 03:16:32.137793 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.130769    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 04 03:16:32.137793 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.131781    2575 server.go:1014] "Starting client certificate rotation"
Nov 04 03:16:32.137793 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.131892    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 03:16:32.137793 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.132794    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 03:16:32.137793 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.133903    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 03:16:32.137793 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.137396    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 03:16:32.151398 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.151381    2575 log.go:25] "Validated CRI v1 runtime API"
Nov 04 03:16:32.154996 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.154982    2575 log.go:25] "Validated CRI v1 image API"
Nov 04 03:16:32.155928 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.155908    2575 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 03:16:32.162281 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.162264    2575 fs.go:135] Filesystem UUIDs: map[4c82457a-93b6-4363-9b2f-48445864594d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a836b55b-aa26-4efc-8c03-ee09c90a57e3:/dev/nvme0n1p4]
Nov 04 03:16:32.162328 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.162282    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 04 03:16:32.169966 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.169854    2575 manager.go:217] Machine: {Timestamp:2026-11-04 03:16:32.168317289 +0000 UTC m=+0.395728459 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101402 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ca891bee4ef787ecbe98ad4d899d3 SystemUUID:ec2ca891-bee4-ef78-7ecb-e98ad4d899d3 BootID:950e00b3-79cb-475d-9ae5-cf1007dbeb72 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:f2:2d:17:31 Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:f2:2d:17:31 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:97:13:c9:eb:e9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 03:16:32.169966 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.169959    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 03:16:32.170108 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.170067    2575 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 03:16:32.171534 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.171510    2575 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 03:16:32.171671 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.171536    2575 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-242.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 03:16:32.172562 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.172552    2575 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 03:16:32.172595 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.172564    2575 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 03:16:32.172595 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.172583    2575 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 03:16:32.172654 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.172597    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 03:16:32.173793 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.173783    2575 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 03:16:32.177815 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.177803    2575 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 03:16:32.177932 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.177923    2575 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 03:16:32.178987 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.178978    2575 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 03:16:32.179018 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.178992    2575 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 03:16:32.179018 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.179002    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 03:16:32.179018 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.179011    2575 kubelet.go:405] "Adding apiserver pod source"
Nov 04 03:16:32.179097 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.179019    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 03:16:32.179380 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.179369    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 03:16:32.182581 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.182560    2575 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 03:16:32.188834 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.188818    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 03:16:32.189711 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.189699    2575 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 03:16:32.189766 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.189729    2575 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 03:16:32.189889 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.189877    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 03:16:32.189928 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.189898    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 03:16:32.189928 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.189907    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 03:16:32.189928 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.189916    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 03:16:32.189928 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.189925    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 03:16:32.190038 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.189933    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 03:16:32.190571 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.190562    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 03:16:32.190602 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.190572    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 03:16:32.190602 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.190580    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 03:16:32.190602 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.190598    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 03:16:32.190676 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.190611    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 03:16:32.190676 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.190620    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 03:16:32.191748 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.191739    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 03:16:32.191748 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.191748    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 03:16:32.195825 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.195805    2575 server.go:1263] "Started kubelet"
Nov 04 03:16:32.195927 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.195868    2575 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 03:16:32.195980 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.195919    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 03:16:32.196012 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.195994    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 03:16:32.196573 ip-10-0-135-242 systemd[1]: Started Kubernetes Kubelet.
Nov 04 03:16:32.197889 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.197868    2575 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 03:16:32.200133 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.200121    2575 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 03:16:32.204747 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:32.204727    2575 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 04 03:16:32.205942 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.205928    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 03:16:32.206590 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.206571    2575 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 03:16:32.206764 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:32.206745    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-242.us-west-2.compute.internal\" not found"
Nov 04 03:16:32.207417 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.207398    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 03:16:32.207417 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.207403    2575 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 03:16:32.207546 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.207427    2575 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 03:16:32.207592 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.207546    2575 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 03:16:32.207592 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.207554    2575 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 03:16:32.207677 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.207617    2575 factory.go:55] Registering systemd factory
Nov 04 03:16:32.207677 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.207634    2575 factory.go:223] Registration of the systemd container factory successfully
Nov 04 03:16:32.207863 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.207849    2575 factory.go:153] Registering CRI-O factory
Nov 04 03:16:32.207863 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.207864    2575 factory.go:223] Registration of the crio container factory successfully
Nov 04 03:16:32.207955 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.207913    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 03:16:32.207955 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.207931    2575 factory.go:103] Registering Raw factory
Nov 04 03:16:32.207955 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.207946    2575 manager.go:1199] Started watching for new ooms in manager
Nov 04 03:16:32.208920 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.208908    2575 manager.go:319] Starting recovery of all containers
Nov 04 03:16:32.220643 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.220623    2575 manager.go:324] Recovery completed
Nov 04 03:16:32.225286 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.225273    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:32.226935 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.226918    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:32.227012 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.226949    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:32.227012 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.226963    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:32.227498 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.227478    2575 cpu_manager.go:226] "Starting" policy="none"
Nov 04 03:16:32.227498 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.227492    2575 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 03:16:32.227595 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.227504    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:32.227595 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.227510    2575 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 03:16:32.228254 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.228238    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:32.228316 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.228264    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:32.228316 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.228273    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:32.229366 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.229349    2575 policy_none.go:50] "Start"
Nov 04 03:16:32.229887 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.229877    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 03:16:32.229923 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.229891    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 03:16:32.231246 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.231235    2575 policy_none.go:44] "Start"
Nov 04 03:16:32.258909 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.258853    2575 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 03:16:32.262833 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.262809    2575 manager.go:341] "Starting Device Plugin manager"
Nov 04 03:16:32.263486 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:32.263466    2575 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 03:16:32.263486 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.263487    2575 server.go:87] "Starting device plugin registration server"
Nov 04 03:16:32.263733 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.263721    2575 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 03:16:32.263788 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.263735    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 03:16:32.263852 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.263839    2575 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 03:16:32.263932 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.263917    2575 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 03:16:32.263932 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.263926    2575 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 03:16:32.264417 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:32.264394    2575 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 03:16:32.264507 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:32.264439    2575 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-242.us-west-2.compute.internal\" not found"
Nov 04 03:16:32.340744 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.340716    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 03:16:32.342732 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.342032    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 03:16:32.342732 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.342056    2575 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 03:16:32.342732 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.342081    2575 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 03:16:32.342732 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:32.342121    2575 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 04 03:16:32.342732 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:32.342351    2575 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-242.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 04 03:16:32.343545 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:32.342259    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-242.us-west-2.compute.internal.18e4b935c55af4ce  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-242.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-242.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 03:16:32.195777742 +0000 UTC m=+0.423188908,LastTimestamp:2026-11-04 03:16:32.195777742 +0000 UTC m=+0.423188908,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-242.us-west-2.compute.internal,}"
Nov 04 03:16:32.363999 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.363942    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:32.364714 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.364694    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:32.364829 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.364728    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:32.364829 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.364744    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:32.364829 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.364772    2575 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.391261 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.391231    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-242.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 04 03:16:32.437519 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.437489    2575 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 03:16:32.437677 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.437546    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s4vhp"
Nov 04 03:16:32.442411 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.442389    2575 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal"]
Nov 04 03:16:32.442526 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.442453    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:32.443394 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.443378    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:32.443495 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.443407    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:32.443495 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.443424    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:32.445551 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.445537    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:32.445804 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.445779    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.445858 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.445836    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:32.446288 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.446274    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:32.446365 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.446298    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:32.446365 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.446308    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:32.446480 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.446454    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:32.446519 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.446496    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:32.446519 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.446512    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:32.448174 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.448150    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.448225 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.448190    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:32.448836 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.448822    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:32.448923 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.448850    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:32.448923 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.448867    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:32.475147 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:32.475129    2575 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-242.us-west-2.compute.internal\" not found" node="ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.478590 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:32.478573    2575 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-242.us-west-2.compute.internal\" not found" node="ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.491961 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.491931    2575 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:32.494061 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.494047    2575 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:32.508936 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.508917    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c879e759b69c22be08682ecf8baecef5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"c879e759b69c22be08682ecf8baecef5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.508992 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.508941    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c879e759b69c22be08682ecf8baecef5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"c879e759b69c22be08682ecf8baecef5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.508992 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.508958    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/172fe89364b7e525fac46ad33508ae4e-config\") pod \"kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"172fe89364b7e525fac46ad33508ae4e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.587001 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.586971    2575 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:32.588383 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.588367    2575 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:32.589639 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.589622    2575 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.589723 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:32.589651    2575 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-242.us-west-2.compute.internal\": node \"ip-10-0-135-242.us-west-2.compute.internal\" not found"
Nov 04 03:16:32.607447 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.607429    2575 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.609087 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.609068    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c879e759b69c22be08682ecf8baecef5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"c879e759b69c22be08682ecf8baecef5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.609185 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.609101    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/172fe89364b7e525fac46ad33508ae4e-config\") pod \"kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"172fe89364b7e525fac46ad33508ae4e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.609185 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.609127    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c879e759b69c22be08682ecf8baecef5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"c879e759b69c22be08682ecf8baecef5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.609279 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.609187    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/172fe89364b7e525fac46ad33508ae4e-config\") pod \"kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"172fe89364b7e525fac46ad33508ae4e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.609279 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.609192    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c879e759b69c22be08682ecf8baecef5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"c879e759b69c22be08682ecf8baecef5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.609279 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.609190    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c879e759b69c22be08682ecf8baecef5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"c879e759b69c22be08682ecf8baecef5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.665496 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.665434    2575 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:16:32.666329 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.666316    2575 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.725059 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.725035    2575 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:16:32.778976 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.778953    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:32.782523 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:32.782359    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:16:34.784646 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:34.784544    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 04 03:16:34.785144 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:34.784699    2575 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 03:16:34.785144 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:34.784724    2575 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 03:16:34.785144 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:34.784750    2575 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 03:16:34.785144 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:34.784785    2575 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://acd60e97cb07342d6a02bc86b8f04d90-7f8ad830d0f07e24.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-135-242.us-west-2.compute.internal.18e4b935c7365ccc\": read tcp 10.0.135.242:48376->3.19.105.97:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-135-242.us-west-2.compute.internal.18e4b935c7365ccc  default   5207 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-242.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-242.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-242.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 03:16:32 +0000 UTC,LastTimestamp:2026-11-04 03:16:32.443391984 +0000 UTC m=+0.670803150,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-242.us-west-2.compute.internal,}"
Nov 04 03:16:34.832692 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:34.832667    2575 apiserver.go:52] "Watching apiserver"
Nov 04 03:16:34.859181 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:34.859143    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 04 03:16:35.095190 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.095090    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 03:11:33 +0000 UTC" deadline="2028-04-21 07:15:03.281723071 +0000 UTC"
Nov 04 03:16:35.095190 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.095162    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12819h58m28.186585792s"
Nov 04 03:16:35.108981 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.108952    2575 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 03:16:35.109248 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.109229    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal","kube-system/global-pull-secret-syncer-qbcj2","kube-system/konnectivity-agent-sd8pw","openshift-multus/multus-hwrqp","openshift-multus/multus-additional-cni-plugins-xrrqq","openshift-network-operator/iptables-alerter-m9zlr","openshift-ovn-kubernetes/ovnkube-node-zfmg5","kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal","openshift-cluster-node-tuning-operator/tuned-bkcnt","openshift-dns/node-resolver-qk4nr","openshift-multus/network-metrics-daemon-lgvq4","openshift-network-diagnostics/network-check-target-6kpt2","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4","openshift-image-registry/node-ca-tdgzf"]
Nov 04 03:16:35.111479 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.111466    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:35.111543 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.111527    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbcj2" podUID="0924a641-d0e7-4cdb-8130-f17a47a4c026"
Nov 04 03:16:35.113321 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.113303    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sd8pw"
Nov 04 03:16:35.115257 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.115241    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.117093 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.117072    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.118906 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.118892    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:16:35.121026 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.121009    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.123088 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.123073    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.125021 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.125002    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:16:35.127090 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.127072    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:35.127183 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.127124    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lgvq4" podUID="23b77dc0-f7c4-4e72-a022-d75d8c8221fd"
Nov 04 03:16:35.129184 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.129168    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:35.129251 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.129221    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kpt2" podUID="b7f9ded5-2460-4131-ab22-85c462aaae48"
Nov 04 03:16:35.131404 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.131389    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.133663 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.133649    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:16:35.161125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.161112    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 03:16:35.165379 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165358    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq5wt\" (UniqueName: \"kubernetes.io/projected/5d04c72f-9d65-429a-ad6c-dc69054ba75e-kube-api-access-pq5wt\") pod \"iptables-alerter-m9zlr\" (UID: \"5d04c72f-9d65-429a-ad6c-dc69054ba75e\") " pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:16:35.165477 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165392    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-cnibin\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.165477 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165415    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-cni-multus\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.165477 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165436    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-daemon-config\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.165605 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165497    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-multus-certs\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.165605 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165537    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-cni-netd\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.165605 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165575    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-kubernetes\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.165711 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165609    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.165711 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165640    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-device-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.165711 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165667    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.165711 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165691    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-systemd\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.165838 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165713    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.165838 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165737    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-cni-bin\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.165838 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165759    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovnkube-config\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.165838 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165779    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysctl-conf\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.165838 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165802    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kmrh\" (UniqueName: \"kubernetes.io/projected/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-kube-api-access-7kmrh\") pod \"network-metrics-daemon-lgvq4\" (UID: \"23b77dc0-f7c4-4e72-a022-d75d8c8221fd\") " pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:35.165838 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165826    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysctl-d\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.166104 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165874    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-lib-modules\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.166104 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165896    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70ed447c-add1-4e78-94fb-6aa3e4389fb5-hosts-file\") pod \"node-resolver-qk4nr\" (UID: \"70ed447c-add1-4e78-94fb-6aa3e4389fb5\") " pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:16:35.166104 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165914    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-socket-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.166104 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165928    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-run-ovn-kubernetes\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.166104 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.165969    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18329af9-cec7-48fd-aeee-18367bb6c078-host\") pod \"node-ca-tdgzf\" (UID: \"18329af9-cec7-48fd-aeee-18367bb6c078\") " pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:16:35.166104 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166003    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-cni-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.166104 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166037    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-etc-kubernetes\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.166104 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166060    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-sys\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.166104 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166084    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:35.166440 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166117    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs\") pod \"network-metrics-daemon-lgvq4\" (UID: \"23b77dc0-f7c4-4e72-a022-d75d8c8221fd\") " pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:35.166440 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166141    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgp4k\" (UniqueName: \"kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k\") pod \"network-check-target-6kpt2\" (UID: \"b7f9ded5-2460-4131-ab22-85c462aaae48\") " pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:35.166440 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166180    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/18329af9-cec7-48fd-aeee-18367bb6c078-serviceca\") pod \"node-ca-tdgzf\" (UID: \"18329af9-cec7-48fd-aeee-18367bb6c078\") " pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:16:35.166440 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166213    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m65mn\" (UniqueName: \"kubernetes.io/projected/18329af9-cec7-48fd-aeee-18367bb6c078-kube-api-access-m65mn\") pod \"node-ca-tdgzf\" (UID: \"18329af9-cec7-48fd-aeee-18367bb6c078\") " pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:16:35.166440 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166256    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e6f6098-4287-4076-a8c5-2e55067dfb87-konnectivity-ca\") pod \"konnectivity-agent-sd8pw\" (UID: \"7e6f6098-4287-4076-a8c5-2e55067dfb87\") " pod="kube-system/konnectivity-agent-sd8pw"
Nov 04 03:16:35.166440 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166292    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7721e70f-2dda-400b-b1a7-d434249efb7e-cni-binary-copy\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.166440 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166315    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.166440 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166343    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-modprobe-d\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.166440 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166357    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-var-lib-kubelet\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.166440 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166374    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-k8s-cni-cncf-io\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.166440 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166396    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-cni-bin\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.166440 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166414    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-slash\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.166873 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166492    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-socket-dir-parent\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.166873 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166506    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-kubelet\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.166873 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166547    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-node-log\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.166873 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166576    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-log-socket\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.166873 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166611    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-env-overrides\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.166873 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166634    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-tuned\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.166873 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166656    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e6f6098-4287-4076-a8c5-2e55067dfb87-agent-certs\") pod \"konnectivity-agent-sd8pw\" (UID: \"7e6f6098-4287-4076-a8c5-2e55067dfb87\") " pod="kube-system/konnectivity-agent-sd8pw"
Nov 04 03:16:35.166873 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166678    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-hostroot\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.166873 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166699    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-run-netns\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.166873 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166722    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96vcz\" (UniqueName: \"kubernetes.io/projected/4db0986b-dec5-4a04-b0f0-ebbc3298f250-kube-api-access-96vcz\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.166873 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166757    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-cnibin\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.166873 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166783    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70ed447c-add1-4e78-94fb-6aa3e4389fb5-tmp-dir\") pod \"node-resolver-qk4nr\" (UID: \"70ed447c-add1-4e78-94fb-6aa3e4389fb5\") " pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:16:35.166873 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166821    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5d04c72f-9d65-429a-ad6c-dc69054ba75e-iptables-alerter-script\") pod \"iptables-alerter-m9zlr\" (UID: \"5d04c72f-9d65-429a-ad6c-dc69054ba75e\") " pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:16:35.166873 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166863    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-netns\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.167455 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166895    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-var-lib-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.167455 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166918    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmbrz\" (UniqueName: \"kubernetes.io/projected/a4fa5e68-1a28-41d2-9680-c79f0d95e361-kube-api-access-vmbrz\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.167455 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166947    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-etc-selinux\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.167455 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166971    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0924a641-d0e7-4cdb-8130-f17a47a4c026-dbus\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:35.167455 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.166994    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.167455 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167019    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-conf-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.167455 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167100    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-systemd-units\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.167455 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167132    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-ovn\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.167455 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167153    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovnkube-script-lib\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.167455 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167178    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-registration-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.167455 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167194    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-sys-fs\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.167455 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167210    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-system-cni-dir\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.167455 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167223    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.167455 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167251    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-etc-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.167455 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167273    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 03:16:35.167455 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167287    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-systemd\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.167455 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167310    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmghw\" (UniqueName: \"kubernetes.io/projected/70ed447c-add1-4e78-94fb-6aa3e4389fb5-kube-api-access-qmghw\") pod \"node-resolver-qk4nr\" (UID: \"70ed447c-add1-4e78-94fb-6aa3e4389fb5\") " pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:16:35.167953 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167333    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d04c72f-9d65-429a-ad6c-dc69054ba75e-host-slash\") pod \"iptables-alerter-m9zlr\" (UID: \"5d04c72f-9d65-429a-ad6c-dc69054ba75e\") " pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:16:35.167953 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167375    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-vpndz\""
Nov 04 03:16:35.167953 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167373    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-system-cni-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.167953 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167402    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdvfm\" (UniqueName: \"kubernetes.io/projected/7721e70f-2dda-400b-b1a7-d434249efb7e-kube-api-access-wdvfm\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.167953 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167417    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovn-node-metrics-cert\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.167953 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167421    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 03:16:35.167953 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167455    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysconfig\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.167953 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167481    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-run\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.167953 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167494    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-host\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.167953 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167513    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16976140-0bb0-4983-bcb7-b2fa2274d24e-tmp\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.167953 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167556    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0924a641-d0e7-4cdb-8130-f17a47a4c026-kubelet-config\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:35.167953 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167586    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-cni-binary-copy\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.167953 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167606    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmw2d\" (UniqueName: \"kubernetes.io/projected/ea8cb5af-3757-47db-840e-728d8d0ad62d-kube-api-access-dmw2d\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.167953 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167628    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-kubelet\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.167953 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.167643    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nnm8\" (UniqueName: \"kubernetes.io/projected/16976140-0bb0-4983-bcb7-b2fa2274d24e-kube-api-access-7nnm8\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.169323 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.169304    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 03:16:35.169398 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.169324    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 03:16:35.169398 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.169354    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-569bn\""
Nov 04 03:16:35.169398 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.169328    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 03:16:35.169398 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.169380    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 03:16:35.170888 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.170873    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 03:16:35.170888 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.170878    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 03:16:35.170973 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.170883    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x6xxb\""
Nov 04 03:16:35.172770 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.172755    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 03:16:35.172999 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.172985    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:16:35.173046 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.172997    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 03:16:35.173046 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.173001    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fbcnm\""
Nov 04 03:16:35.174812 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.174797    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 03:16:35.174867 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.174797    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 03:16:35.174867 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.174804    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 03:16:35.175016 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.175004    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-56wj9\""
Nov 04 03:16:35.214812 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.214785    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 03:16:35.215058 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.215042    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 03:16:35.215058 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.215051    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 03:16:35.216691 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.216671    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:16:35.216691 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.216677    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 03:16:35.216837 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.216749    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 03:16:35.220401 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.220384    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h66cx\""
Nov 04 03:16:35.263382 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.263363    2575 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 03:16:35.263459 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.263414    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8tw65"
Nov 04 03:16:35.267839 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.267822    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 03:16:35.267937 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.267873    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 03:16:35.267937 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.267885    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 03:16:35.267937 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.267897    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 03:16:35.267937 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.267905    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4vrb5\""
Nov 04 03:16:35.268114 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.267895    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-sys\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.268114 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.267961    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 03:16:35.268114 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.267982    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:35.268114 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268007    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 03:16:35.268114 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268012    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9cxzk\""
Nov 04 03:16:35.268114 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268011    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-sys\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.268114 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268027    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs\") pod \"network-metrics-daemon-lgvq4\" (UID: \"23b77dc0-f7c4-4e72-a022-d75d8c8221fd\") " pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:35.268114 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.267931    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 03:16:35.268114 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268055    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z2nst\""
Nov 04 03:16:35.268114 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268055    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgp4k\" (UniqueName: \"kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k\") pod \"network-check-target-6kpt2\" (UID: \"b7f9ded5-2460-4131-ab22-85c462aaae48\") " pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:35.268114 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.268085    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:35.268114 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268095    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/18329af9-cec7-48fd-aeee-18367bb6c078-serviceca\") pod \"node-ca-tdgzf\" (UID: \"18329af9-cec7-48fd-aeee-18367bb6c078\") " pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:16:35.268636 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268119    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m65mn\" (UniqueName: \"kubernetes.io/projected/18329af9-cec7-48fd-aeee-18367bb6c078-kube-api-access-m65mn\") pod \"node-ca-tdgzf\" (UID: \"18329af9-cec7-48fd-aeee-18367bb6c078\") " pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:16:35.268636 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.268085    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:35.268636 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.268152    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs podName:23b77dc0-f7c4-4e72-a022-d75d8c8221fd nodeName:}" failed. No retries permitted until 2026-11-04 03:16:35.768120717 +0000 UTC m=+2.342851284 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs") pod "network-metrics-daemon-lgvq4" (UID: "23b77dc0-f7c4-4e72-a022-d75d8c8221fd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:35.268636 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.268183    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret podName:0924a641-d0e7-4cdb-8130-f17a47a4c026 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:35.768167087 +0000 UTC m=+2.342897640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret") pod "global-pull-secret-syncer-qbcj2" (UID: "0924a641-d0e7-4cdb-8130-f17a47a4c026") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:35.268636 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268204    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e6f6098-4287-4076-a8c5-2e55067dfb87-konnectivity-ca\") pod \"konnectivity-agent-sd8pw\" (UID: \"7e6f6098-4287-4076-a8c5-2e55067dfb87\") " pod="kube-system/konnectivity-agent-sd8pw"
Nov 04 03:16:35.268636 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268230    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7721e70f-2dda-400b-b1a7-d434249efb7e-cni-binary-copy\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.268636 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268248    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.268636 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268265    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-modprobe-d\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.268636 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268281    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-var-lib-kubelet\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.268636 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268305    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-k8s-cni-cncf-io\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.268636 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268322    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.268636 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268328    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-cni-bin\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.268636 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268364    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-var-lib-kubelet\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.268636 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268371    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-slash\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.268636 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268398    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-socket-dir-parent\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.268636 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268406    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-modprobe-d\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268372    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-k8s-cni-cncf-io\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268427    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-kubelet\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268405    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-slash\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268438    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-socket-dir-parent\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268401    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-cni-bin\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268494    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-node-log\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268498    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-kubelet\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268517    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-log-socket\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268534    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-node-log\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268541    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-env-overrides\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268549    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/18329af9-cec7-48fd-aeee-18367bb6c078-serviceca\") pod \"node-ca-tdgzf\" (UID: \"18329af9-cec7-48fd-aeee-18367bb6c078\") " pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268541    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-log-socket\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268576    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-tuned\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268609    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e6f6098-4287-4076-a8c5-2e55067dfb87-agent-certs\") pod \"konnectivity-agent-sd8pw\" (UID: \"7e6f6098-4287-4076-a8c5-2e55067dfb87\") " pod="kube-system/konnectivity-agent-sd8pw"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268631    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-hostroot\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268658    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-run-netns\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268701    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96vcz\" (UniqueName: \"kubernetes.io/projected/4db0986b-dec5-4a04-b0f0-ebbc3298f250-kube-api-access-96vcz\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.269400 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268720    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-hostroot\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268727    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-cnibin\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268766    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70ed447c-add1-4e78-94fb-6aa3e4389fb5-tmp-dir\") pod \"node-resolver-qk4nr\" (UID: \"70ed447c-add1-4e78-94fb-6aa3e4389fb5\") " pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268770    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-cnibin\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268795    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5d04c72f-9d65-429a-ad6c-dc69054ba75e-iptables-alerter-script\") pod \"iptables-alerter-m9zlr\" (UID: \"5d04c72f-9d65-429a-ad6c-dc69054ba75e\") " pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268821    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-netns\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268861    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-var-lib-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268834    2575 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268889    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmbrz\" (UniqueName: \"kubernetes.io/projected/a4fa5e68-1a28-41d2-9680-c79f0d95e361-kube-api-access-vmbrz\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268912    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-etc-selinux\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268929    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7721e70f-2dda-400b-b1a7-d434249efb7e-cni-binary-copy\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268940    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0924a641-d0e7-4cdb-8130-f17a47a4c026-dbus\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268949    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-run-netns\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268964    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268988    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-conf-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269013    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-systemd-units\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269025    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-netns\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269039    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-ovn\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.270219 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269072    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0924a641-d0e7-4cdb-8130-f17a47a4c026-dbus\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:35.271055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269085    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-ovn\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.271055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.268914    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e6f6098-4287-4076-a8c5-2e55067dfb87-konnectivity-ca\") pod \"konnectivity-agent-sd8pw\" (UID: \"7e6f6098-4287-4076-a8c5-2e55067dfb87\") " pod="kube-system/konnectivity-agent-sd8pw"
Nov 04 03:16:35.271055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269113    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovnkube-script-lib\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.271055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269141    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-systemd-units\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.271055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269144    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-registration-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.271055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269155    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-conf-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.271055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269013    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-env-overrides\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.271055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269091    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-etc-selinux\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.271055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269226    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-var-lib-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.271055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269275    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-registration-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.271055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269281    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.271055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269299    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-sys-fs\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.271055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269323    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-system-cni-dir\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.271055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269344    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.271055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269364    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-etc-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.271055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269376    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-system-cni-dir\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.271055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269383    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-systemd\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269404    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmghw\" (UniqueName: \"kubernetes.io/projected/70ed447c-add1-4e78-94fb-6aa3e4389fb5-kube-api-access-qmghw\") pod \"node-resolver-qk4nr\" (UID: \"70ed447c-add1-4e78-94fb-6aa3e4389fb5\") " pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269408    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-sys-fs\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269433    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d04c72f-9d65-429a-ad6c-dc69054ba75e-host-slash\") pod \"iptables-alerter-m9zlr\" (UID: \"5d04c72f-9d65-429a-ad6c-dc69054ba75e\") " pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269436    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70ed447c-add1-4e78-94fb-6aa3e4389fb5-tmp-dir\") pod \"node-resolver-qk4nr\" (UID: \"70ed447c-add1-4e78-94fb-6aa3e4389fb5\") " pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269457    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-system-cni-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269463    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5d04c72f-9d65-429a-ad6c-dc69054ba75e-iptables-alerter-script\") pod \"iptables-alerter-m9zlr\" (UID: \"5d04c72f-9d65-429a-ad6c-dc69054ba75e\") " pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269518    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-system-cni-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269617    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdvfm\" (UniqueName: \"kubernetes.io/projected/7721e70f-2dda-400b-b1a7-d434249efb7e-kube-api-access-wdvfm\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269641    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovn-node-metrics-cert\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269633    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-etc-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269664    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovnkube-script-lib\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269671    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d04c72f-9d65-429a-ad6c-dc69054ba75e-host-slash\") pod \"iptables-alerter-m9zlr\" (UID: \"5d04c72f-9d65-429a-ad6c-dc69054ba75e\") " pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269675    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysconfig\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269671    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-systemd\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269696    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-run\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269717    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-host\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269725    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysconfig\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.271524 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269736    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16976140-0bb0-4983-bcb7-b2fa2274d24e-tmp\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.272125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269794    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0924a641-d0e7-4cdb-8130-f17a47a4c026-kubelet-config\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:35.272125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269803    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-host\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.272125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269822    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-cni-binary-copy\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.272125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269901    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.272125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269942    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-run\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.272125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269943    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0924a641-d0e7-4cdb-8130-f17a47a4c026-kubelet-config\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:35.272125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269969    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmw2d\" (UniqueName: \"kubernetes.io/projected/ea8cb5af-3757-47db-840e-728d8d0ad62d-kube-api-access-dmw2d\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.272125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.269997    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-kubelet\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.272125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270025    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nnm8\" (UniqueName: \"kubernetes.io/projected/16976140-0bb0-4983-bcb7-b2fa2274d24e-kube-api-access-7nnm8\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.272125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270046    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-kubelet\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.272125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270053    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq5wt\" (UniqueName: \"kubernetes.io/projected/5d04c72f-9d65-429a-ad6c-dc69054ba75e-kube-api-access-pq5wt\") pod \"iptables-alerter-m9zlr\" (UID: \"5d04c72f-9d65-429a-ad6c-dc69054ba75e\") " pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:16:35.272125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270076    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-cnibin\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.272125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270100    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-cni-multus\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.272125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270125    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-daemon-config\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.272125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270149    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-multus-certs\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.272125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270173    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-cni-netd\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.272125 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270198    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-kubernetes\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.272799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270223    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.272799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270232    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-cni-multus\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.272799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270174    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-cnibin\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.272799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270250    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-device-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.272799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270270    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-cni-binary-copy\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.272799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270286    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.272799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270294    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-device-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.272799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270311    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-systemd\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.272799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270334    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.272799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270334    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-kubernetes\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.272799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270347    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.272799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270356    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-cni-bin\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.272799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270365    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-cni-netd\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.272799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270369    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-systemd\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.272799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270383    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-cni-bin\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.272799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270394    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-multus-certs\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.272799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270413    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270426    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovnkube-config\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270446    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysctl-conf\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270480    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kmrh\" (UniqueName: \"kubernetes.io/projected/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-kube-api-access-7kmrh\") pod \"network-metrics-daemon-lgvq4\" (UID: \"23b77dc0-f7c4-4e72-a022-d75d8c8221fd\") " pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270497    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysctl-d\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270512    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-lib-modules\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270530    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70ed447c-add1-4e78-94fb-6aa3e4389fb5-hosts-file\") pod \"node-resolver-qk4nr\" (UID: \"70ed447c-add1-4e78-94fb-6aa3e4389fb5\") " pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270554    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-socket-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270576    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-run-ovn-kubernetes\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270599    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18329af9-cec7-48fd-aeee-18367bb6c078-host\") pod \"node-ca-tdgzf\" (UID: \"18329af9-cec7-48fd-aeee-18367bb6c078\") " pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270624    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-cni-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270651    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-etc-kubernetes\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270722    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-etc-kubernetes\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270757    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-run-ovn-kubernetes\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270810    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysctl-conf\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270827    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-cni-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270834    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18329af9-cec7-48fd-aeee-18367bb6c078-host\") pod \"node-ca-tdgzf\" (UID: \"18329af9-cec7-48fd-aeee-18367bb6c078\") " pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270906    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70ed447c-add1-4e78-94fb-6aa3e4389fb5-hosts-file\") pod \"node-resolver-qk4nr\" (UID: \"70ed447c-add1-4e78-94fb-6aa3e4389fb5\") " pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:16:35.273624 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270918    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-lib-modules\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.274506 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.270947    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysctl-d\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.274506 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.271096    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-socket-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.274506 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.271951    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-daemon-config\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.274506 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.272140    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovnkube-config\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.274506 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.272150    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.274506 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.272668    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-tuned\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.274506 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.272715    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16976140-0bb0-4983-bcb7-b2fa2274d24e-tmp\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.274506 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.272813    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e6f6098-4287-4076-a8c5-2e55067dfb87-agent-certs\") pod \"konnectivity-agent-sd8pw\" (UID: \"7e6f6098-4287-4076-a8c5-2e55067dfb87\") " pod="kube-system/konnectivity-agent-sd8pw"
Nov 04 03:16:35.274506 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.272929    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovn-node-metrics-cert\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.331527 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.331479    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:35.331527 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.331514    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:35.331527 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.331533    2575 projected.go:196] Error preparing data for projected volume kube-api-access-hgp4k for pod openshift-network-diagnostics/network-check-target-6kpt2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:35.331802 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.331600    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k podName:b7f9ded5-2460-4131-ab22-85c462aaae48 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:35.831582546 +0000 UTC m=+2.406313101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hgp4k" (UniqueName: "kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k") pod "network-check-target-6kpt2" (UID: "b7f9ded5-2460-4131-ab22-85c462aaae48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:35.333917 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.333879    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nnm8\" (UniqueName: \"kubernetes.io/projected/16976140-0bb0-4983-bcb7-b2fa2274d24e-kube-api-access-7nnm8\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.334072 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.333957    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96vcz\" (UniqueName: \"kubernetes.io/projected/4db0986b-dec5-4a04-b0f0-ebbc3298f250-kube-api-access-96vcz\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.334216 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.334189    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m65mn\" (UniqueName: \"kubernetes.io/projected/18329af9-cec7-48fd-aeee-18367bb6c078-kube-api-access-m65mn\") pod \"node-ca-tdgzf\" (UID: \"18329af9-cec7-48fd-aeee-18367bb6c078\") " pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:16:35.334341 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.334324    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq5wt\" (UniqueName: \"kubernetes.io/projected/5d04c72f-9d65-429a-ad6c-dc69054ba75e-kube-api-access-pq5wt\") pod \"iptables-alerter-m9zlr\" (UID: \"5d04c72f-9d65-429a-ad6c-dc69054ba75e\") " pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:16:35.334404 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.334356    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmghw\" (UniqueName: \"kubernetes.io/projected/70ed447c-add1-4e78-94fb-6aa3e4389fb5-kube-api-access-qmghw\") pod \"node-resolver-qk4nr\" (UID: \"70ed447c-add1-4e78-94fb-6aa3e4389fb5\") " pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:16:35.334404 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.334382    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmw2d\" (UniqueName: \"kubernetes.io/projected/ea8cb5af-3757-47db-840e-728d8d0ad62d-kube-api-access-dmw2d\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.338684 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.338659    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmbrz\" (UniqueName: \"kubernetes.io/projected/a4fa5e68-1a28-41d2-9680-c79f0d95e361-kube-api-access-vmbrz\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.338773 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.338684    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdvfm\" (UniqueName: \"kubernetes.io/projected/7721e70f-2dda-400b-b1a7-d434249efb7e-kube-api-access-wdvfm\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.339274 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.339251    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kmrh\" (UniqueName: \"kubernetes.io/projected/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-kube-api-access-7kmrh\") pod \"network-metrics-daemon-lgvq4\" (UID: \"23b77dc0-f7c4-4e72-a022-d75d8c8221fd\") " pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:35.445330 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.445255    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sd8pw"
Nov 04 03:16:35.452096 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.452077    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hwrqp"
Nov 04 03:16:35.495734 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.495698    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:16:35.501716 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.501699    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:16:35.517048 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.517024    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:35.521639 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.521622    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:16:35.536938 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.536915    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:16:35.557501 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.557481    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:16:35.559635 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.559620    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:16:35.614647 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:35.614618    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc879e759b69c22be08682ecf8baecef5.slice/crio-e27f857d8e36a049a3a9bdd3de851a12ca9a6447a9ea73fcedff7ce387d215d6 WatchSource:0}: Error finding container e27f857d8e36a049a3a9bdd3de851a12ca9a6447a9ea73fcedff7ce387d215d6: Status 404 returned error can't find the container with id e27f857d8e36a049a3a9bdd3de851a12ca9a6447a9ea73fcedff7ce387d215d6
Nov 04 03:16:35.620344 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.620328    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:16:35.692528 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:35.692498    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod172fe89364b7e525fac46ad33508ae4e.slice/crio-f33e2afd49a996b53b9e3356f66dea2f35333eb4387d664a2f2d30467fb8b42e WatchSource:0}: Error finding container f33e2afd49a996b53b9e3356f66dea2f35333eb4387d664a2f2d30467fb8b42e: Status 404 returned error can't find the container with id f33e2afd49a996b53b9e3356f66dea2f35333eb4387d664a2f2d30467fb8b42e
Nov 04 03:16:35.756244 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.756220    2575 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:35.773734 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.773706    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:35.773867 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.773738    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs\") pod \"network-metrics-daemon-lgvq4\" (UID: \"23b77dc0-f7c4-4e72-a022-d75d8c8221fd\") " pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:35.773867 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.773853    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:35.773936 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.773908    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret podName:0924a641-d0e7-4cdb-8130-f17a47a4c026 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:36.773894348 +0000 UTC m=+3.348624890 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret") pod "global-pull-secret-syncer-qbcj2" (UID: "0924a641-d0e7-4cdb-8130-f17a47a4c026") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:35.773936 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.773860    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:35.774001 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.773941    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs podName:23b77dc0-f7c4-4e72-a022-d75d8c8221fd nodeName:}" failed. No retries permitted until 2026-11-04 03:16:36.77393475 +0000 UTC m=+3.348665320 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs") pod "network-metrics-daemon-lgvq4" (UID: "23b77dc0-f7c4-4e72-a022-d75d8c8221fd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:35.874854 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.874811    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgp4k\" (UniqueName: \"kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k\") pod \"network-check-target-6kpt2\" (UID: \"b7f9ded5-2460-4131-ab22-85c462aaae48\") " pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:35.875359 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.874945    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:35.875359 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.874960    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:35.875359 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.874969    2575 projected.go:196] Error preparing data for projected volume kube-api-access-hgp4k for pod openshift-network-diagnostics/network-check-target-6kpt2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:35.875359 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:35.875012    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k podName:b7f9ded5-2460-4131-ab22-85c462aaae48 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:36.875001003 +0000 UTC m=+3.449731541 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hgp4k" (UniqueName: "kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k") pod "network-check-target-6kpt2" (UID: "b7f9ded5-2460-4131-ab22-85c462aaae48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:35.998944 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.998835    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal" event={"ID":"172fe89364b7e525fac46ad33508ae4e","Type":"ContainerStarted","Data":"f33e2afd49a996b53b9e3356f66dea2f35333eb4387d664a2f2d30467fb8b42e"}
Nov 04 03:16:35.999908 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:35.999875    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal" event={"ID":"c879e759b69c22be08682ecf8baecef5","Type":"ContainerStarted","Data":"e27f857d8e36a049a3a9bdd3de851a12ca9a6447a9ea73fcedff7ce387d215d6"}
Nov 04 03:16:36.210181 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:36.210156    2575 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:36.246471 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:36.246438    2575 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:36.266634 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:36.266573    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 03:11:34 +0000 UTC" deadline="2028-05-20 03:07:57.334909552 +0000 UTC"
Nov 04 03:16:36.266634 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:36.266596    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13511h51m21.068316306s"
Nov 04 03:16:36.339833 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:36.339807    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea8cb5af_3757_47db_840e_728d8d0ad62d.slice/crio-7cbf476f2b9749722e3dcf3981c9f9728e3bcd31d5d1bbb7747647cf8bd15fcc WatchSource:0}: Error finding container 7cbf476f2b9749722e3dcf3981c9f9728e3bcd31d5d1bbb7747647cf8bd15fcc: Status 404 returned error can't find the container with id 7cbf476f2b9749722e3dcf3981c9f9728e3bcd31d5d1bbb7747647cf8bd15fcc
Nov 04 03:16:36.384706 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:36.384674    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e6f6098_4287_4076_a8c5_2e55067dfb87.slice/crio-bfc3a0432cc1469411ac6bd53ec3a564216e325b84a56bfb8341561fdba04e53 WatchSource:0}: Error finding container bfc3a0432cc1469411ac6bd53ec3a564216e325b84a56bfb8341561fdba04e53: Status 404 returned error can't find the container with id bfc3a0432cc1469411ac6bd53ec3a564216e325b84a56bfb8341561fdba04e53
Nov 04 03:16:36.440794 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:36.440769    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4fa5e68_1a28_41d2_9680_c79f0d95e361.slice/crio-0e16d135de0117c644a6f9c809237a412d553e7ed6b63de6fc9e9d168c32f20a WatchSource:0}: Error finding container 0e16d135de0117c644a6f9c809237a412d553e7ed6b63de6fc9e9d168c32f20a: Status 404 returned error can't find the container with id 0e16d135de0117c644a6f9c809237a412d553e7ed6b63de6fc9e9d168c32f20a
Nov 04 03:16:36.444407 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:36.444385    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70ed447c_add1_4e78_94fb_6aa3e4389fb5.slice/crio-ff7a8776f992f8a0d459f0df3ca7b2661e866cf80564a52c87c90d9cef4b5289 WatchSource:0}: Error finding container ff7a8776f992f8a0d459f0df3ca7b2661e866cf80564a52c87c90d9cef4b5289: Status 404 returned error can't find the container with id ff7a8776f992f8a0d459f0df3ca7b2661e866cf80564a52c87c90d9cef4b5289
Nov 04 03:16:36.457043 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:36.457018    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7721e70f_2dda_400b_b1a7_d434249efb7e.slice/crio-8ff1b16ada0753c352d1a960e89f472017e18d1987395cfc7a61a749576572f2 WatchSource:0}: Error finding container 8ff1b16ada0753c352d1a960e89f472017e18d1987395cfc7a61a749576572f2: Status 404 returned error can't find the container with id 8ff1b16ada0753c352d1a960e89f472017e18d1987395cfc7a61a749576572f2
Nov 04 03:16:36.463756 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:36.463725    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16976140_0bb0_4983_bcb7_b2fa2274d24e.slice/crio-6febd61d2ee5c972dbe2099abf7d8c29db3ed5301a9228fe96936677742028bb WatchSource:0}: Error finding container 6febd61d2ee5c972dbe2099abf7d8c29db3ed5301a9228fe96936677742028bb: Status 404 returned error can't find the container with id 6febd61d2ee5c972dbe2099abf7d8c29db3ed5301a9228fe96936677742028bb
Nov 04 03:16:36.464480 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:36.464453    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18329af9_cec7_48fd_aeee_18367bb6c078.slice/crio-ae0dca1e7493e5e505cebb8085274e51457576c0988a47c3ae240f9702f9879f WatchSource:0}: Error finding container ae0dca1e7493e5e505cebb8085274e51457576c0988a47c3ae240f9702f9879f: Status 404 returned error can't find the container with id ae0dca1e7493e5e505cebb8085274e51457576c0988a47c3ae240f9702f9879f
Nov 04 03:16:36.483414 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:36.483395    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4db0986b_dec5_4a04_b0f0_ebbc3298f250.slice/crio-a6288b478b499a7b2d1ab2950b3bd519717726b9c9276dd9d8ce055b7105d94d WatchSource:0}: Error finding container a6288b478b499a7b2d1ab2950b3bd519717726b9c9276dd9d8ce055b7105d94d: Status 404 returned error can't find the container with id a6288b478b499a7b2d1ab2950b3bd519717726b9c9276dd9d8ce055b7105d94d
Nov 04 03:16:36.506304 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:16:36.506286    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d04c72f_9d65_429a_ad6c_dc69054ba75e.slice/crio-61aede999be255b0ba384dc4cd57becbdfacbd165dfedb10e772c35f813819fd WatchSource:0}: Error finding container 61aede999be255b0ba384dc4cd57becbdfacbd165dfedb10e772c35f813819fd: Status 404 returned error can't find the container with id 61aede999be255b0ba384dc4cd57becbdfacbd165dfedb10e772c35f813819fd
Nov 04 03:16:36.784688 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:36.784651    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:36.784887 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:36.784698    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs\") pod \"network-metrics-daemon-lgvq4\" (UID: \"23b77dc0-f7c4-4e72-a022-d75d8c8221fd\") " pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:36.784887 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:36.784824    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:36.785017 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:36.784892    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs podName:23b77dc0-f7c4-4e72-a022-d75d8c8221fd nodeName:}" failed. No retries permitted until 2026-11-04 03:16:38.784874218 +0000 UTC m=+5.359604757 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs") pod "network-metrics-daemon-lgvq4" (UID: "23b77dc0-f7c4-4e72-a022-d75d8c8221fd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:36.785017 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:36.784958    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:36.785017 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:36.784995    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret podName:0924a641-d0e7-4cdb-8130-f17a47a4c026 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:38.784983302 +0000 UTC m=+5.359713855 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret") pod "global-pull-secret-syncer-qbcj2" (UID: "0924a641-d0e7-4cdb-8130-f17a47a4c026") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:36.885720 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:36.885658    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgp4k\" (UniqueName: \"kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k\") pod \"network-check-target-6kpt2\" (UID: \"b7f9ded5-2460-4131-ab22-85c462aaae48\") " pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:36.886239 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:36.885807    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:36.886239 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:36.885831    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:36.886239 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:36.885881    2575 projected.go:196] Error preparing data for projected volume kube-api-access-hgp4k for pod openshift-network-diagnostics/network-check-target-6kpt2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:36.886239 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:36.885929    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k podName:b7f9ded5-2460-4131-ab22-85c462aaae48 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:38.885915494 +0000 UTC m=+5.460646045 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hgp4k" (UniqueName: "kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k") pod "network-check-target-6kpt2" (UID: "b7f9ded5-2460-4131-ab22-85c462aaae48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:36.995887 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:36.995839    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:36.995887 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:36.995888    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:36.996131 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:36.995972    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:36.996131 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:36.995989    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbcj2" podUID="0924a641-d0e7-4cdb-8130-f17a47a4c026"
Nov 04 03:16:36.996131 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:36.996082    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lgvq4" podUID="23b77dc0-f7c4-4e72-a022-d75d8c8221fd"
Nov 04 03:16:36.996287 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:36.996204    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kpt2" podUID="b7f9ded5-2460-4131-ab22-85c462aaae48"
Nov 04 03:16:37.003797 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:37.003767    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qk4nr" event={"ID":"70ed447c-add1-4e78-94fb-6aa3e4389fb5","Type":"ContainerStarted","Data":"ff7a8776f992f8a0d459f0df3ca7b2661e866cf80564a52c87c90d9cef4b5289"}
Nov 04 03:16:37.006146 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:37.006119    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5" event={"ID":"a4fa5e68-1a28-41d2-9680-c79f0d95e361","Type":"ContainerStarted","Data":"0e16d135de0117c644a6f9c809237a412d553e7ed6b63de6fc9e9d168c32f20a"}
Nov 04 03:16:37.008577 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:37.008551    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sd8pw" event={"ID":"7e6f6098-4287-4076-a8c5-2e55067dfb87","Type":"ContainerStarted","Data":"bfc3a0432cc1469411ac6bd53ec3a564216e325b84a56bfb8341561fdba04e53"}
Nov 04 03:16:37.010336 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:37.010312    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrrqq" event={"ID":"ea8cb5af-3757-47db-840e-728d8d0ad62d","Type":"ContainerStarted","Data":"7cbf476f2b9749722e3dcf3981c9f9728e3bcd31d5d1bbb7747647cf8bd15fcc"}
Nov 04 03:16:37.012196 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:37.012173    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4" event={"ID":"4db0986b-dec5-4a04-b0f0-ebbc3298f250","Type":"ContainerStarted","Data":"a6288b478b499a7b2d1ab2950b3bd519717726b9c9276dd9d8ce055b7105d94d"}
Nov 04 03:16:37.013898 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:37.013873    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tdgzf" event={"ID":"18329af9-cec7-48fd-aeee-18367bb6c078","Type":"ContainerStarted","Data":"ae0dca1e7493e5e505cebb8085274e51457576c0988a47c3ae240f9702f9879f"}
Nov 04 03:16:37.016199 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:37.016178    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bkcnt" event={"ID":"16976140-0bb0-4983-bcb7-b2fa2274d24e","Type":"ContainerStarted","Data":"6febd61d2ee5c972dbe2099abf7d8c29db3ed5301a9228fe96936677742028bb"}
Nov 04 03:16:37.017534 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:37.017490    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m9zlr" event={"ID":"5d04c72f-9d65-429a-ad6c-dc69054ba75e","Type":"ContainerStarted","Data":"61aede999be255b0ba384dc4cd57becbdfacbd165dfedb10e772c35f813819fd"}
Nov 04 03:16:37.019079 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:37.019057    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hwrqp" event={"ID":"7721e70f-2dda-400b-b1a7-d434249efb7e","Type":"ContainerStarted","Data":"8ff1b16ada0753c352d1a960e89f472017e18d1987395cfc7a61a749576572f2"}
Nov 04 03:16:37.266821 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:37.266761    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 03:11:34 +0000 UTC" deadline="2028-04-19 01:12:38.796017742 +0000 UTC"
Nov 04 03:16:37.266969 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:37.266825    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12765h56m1.529197143s"
Nov 04 03:16:38.805530 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:38.805221    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:38.805530 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:38.805255    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs\") pod \"network-metrics-daemon-lgvq4\" (UID: \"23b77dc0-f7c4-4e72-a022-d75d8c8221fd\") " pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:38.805530 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:38.805364    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:38.805530 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:38.805424    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs podName:23b77dc0-f7c4-4e72-a022-d75d8c8221fd nodeName:}" failed. No retries permitted until 2026-11-04 03:16:42.805402357 +0000 UTC m=+9.380132910 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs") pod "network-metrics-daemon-lgvq4" (UID: "23b77dc0-f7c4-4e72-a022-d75d8c8221fd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:38.805530 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:38.805480    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:38.805530 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:38.805504    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret podName:0924a641-d0e7-4cdb-8130-f17a47a4c026 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:42.805497046 +0000 UTC m=+9.380227584 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret") pod "global-pull-secret-syncer-qbcj2" (UID: "0924a641-d0e7-4cdb-8130-f17a47a4c026") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:38.905763 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:38.905728    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgp4k\" (UniqueName: \"kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k\") pod \"network-check-target-6kpt2\" (UID: \"b7f9ded5-2460-4131-ab22-85c462aaae48\") " pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:38.905998 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:38.905952    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:38.905998 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:38.905977    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:38.905998 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:38.905989    2575 projected.go:196] Error preparing data for projected volume kube-api-access-hgp4k for pod openshift-network-diagnostics/network-check-target-6kpt2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:38.906165 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:38.906057    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k podName:b7f9ded5-2460-4131-ab22-85c462aaae48 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:42.906035728 +0000 UTC m=+9.480766269 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hgp4k" (UniqueName: "kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k") pod "network-check-target-6kpt2" (UID: "b7f9ded5-2460-4131-ab22-85c462aaae48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:38.995147 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:38.995075    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:38.995311 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:38.995194    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kpt2" podUID="b7f9ded5-2460-4131-ab22-85c462aaae48"
Nov 04 03:16:38.995311 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:38.995251    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:38.995428 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:38.995321    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbcj2" podUID="0924a641-d0e7-4cdb-8130-f17a47a4c026"
Nov 04 03:16:38.995428 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:38.995362    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:38.995428 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:38.995424    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lgvq4" podUID="23b77dc0-f7c4-4e72-a022-d75d8c8221fd"
Nov 04 03:16:40.060541 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:40.060304    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal" event={"ID":"172fe89364b7e525fac46ad33508ae4e","Type":"ContainerStarted","Data":"13b5785c350ae39cb3006159e1ce9b68622ca910b214075c5b036526285477d9"}
Nov 04 03:16:40.067327 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:40.067291    2575 generic.go:356] "Generic (PLEG): container finished" podID="c879e759b69c22be08682ecf8baecef5" containerID="6c28860e1a7a3a18ef27cb6ff5b1b979164e19954395260149d86f903f773fc2" exitCode=0
Nov 04 03:16:40.067469 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:40.067347    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal" event={"ID":"c879e759b69c22be08682ecf8baecef5","Type":"ContainerDied","Data":"6c28860e1a7a3a18ef27cb6ff5b1b979164e19954395260149d86f903f773fc2"}
Nov 04 03:16:40.178347 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:40.178286    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal" podStartSLOduration=6.178266873 podStartE2EDuration="6.178266873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:40.177745002 +0000 UTC m=+6.752475564" watchObservedRunningTime="2026-11-04 03:16:40.178266873 +0000 UTC m=+6.752997446"
Nov 04 03:16:40.996038 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:40.995736    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:40.996038 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:40.995986    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:40.996038 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:40.996019    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:40.996327 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:40.996093    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lgvq4" podUID="23b77dc0-f7c4-4e72-a022-d75d8c8221fd"
Nov 04 03:16:40.996327 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:40.996174    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbcj2" podUID="0924a641-d0e7-4cdb-8130-f17a47a4c026"
Nov 04 03:16:40.996535 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:40.996489    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kpt2" podUID="b7f9ded5-2460-4131-ab22-85c462aaae48"
Nov 04 03:16:41.076685 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:41.076648    2575 generic.go:356] "Generic (PLEG): container finished" podID="ea8cb5af-3757-47db-840e-728d8d0ad62d" containerID="a96ea1b95b1b9bb184ab07ecbf10578cae2b3af081265251a2b37ea401b00f1e" exitCode=0
Nov 04 03:16:41.077172 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:41.076750    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrrqq" event={"ID":"ea8cb5af-3757-47db-840e-728d8d0ad62d","Type":"ContainerDied","Data":"a96ea1b95b1b9bb184ab07ecbf10578cae2b3af081265251a2b37ea401b00f1e"}
Nov 04 03:16:41.085188 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:41.085164    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal" event={"ID":"c879e759b69c22be08682ecf8baecef5","Type":"ContainerStarted","Data":"0c7e801b95c707f52283fc36c1610f5e7baf75826f024b65c68e1e094d98fb4c"}
Nov 04 03:16:41.373331 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:41.373099    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal" podStartSLOduration=7.3730791 podStartE2EDuration="7.3730791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:41.372165901 +0000 UTC m=+7.946896463" watchObservedRunningTime="2026-11-04 03:16:41.3730791 +0000 UTC m=+7.947809661"
Nov 04 03:16:42.091120 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:42.091049    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sd8pw" event={"ID":"7e6f6098-4287-4076-a8c5-2e55067dfb87","Type":"ContainerStarted","Data":"dcd1ec4e60e93e0ecc7862d971ad12c81188869203a49443671f9e837adec238"}
Nov 04 03:16:42.209941 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:42.209515    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-sd8pw" podStartSLOduration=3.587722744 podStartE2EDuration="8.209496263s" totalImagesPullingTime="4.621773519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:16:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:42.20853584 +0000 UTC m=+8.783266417" watchObservedRunningTime="2026-11-04 03:16:42.209496263 +0000 UTC m=+8.784226823"
Nov 04 03:16:42.363109 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:42.362812    2575 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-sd8pw"
Nov 04 03:16:42.364088 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:42.363625    2575 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-sd8pw"
Nov 04 03:16:42.844598 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:42.844565    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:42.844696 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:42.844635    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs\") pod \"network-metrics-daemon-lgvq4\" (UID: \"23b77dc0-f7c4-4e72-a022-d75d8c8221fd\") " pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:42.844819 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:42.844804    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:42.844889 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:42.844877    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs podName:23b77dc0-f7c4-4e72-a022-d75d8c8221fd nodeName:}" failed. No retries permitted until 2026-11-04 03:16:50.844858662 +0000 UTC m=+17.419589203 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs") pod "network-metrics-daemon-lgvq4" (UID: "23b77dc0-f7c4-4e72-a022-d75d8c8221fd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:42.844961 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:42.844951    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:42.845011 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:42.844985    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret podName:0924a641-d0e7-4cdb-8130-f17a47a4c026 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:50.84497388 +0000 UTC m=+17.419704417 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret") pod "global-pull-secret-syncer-qbcj2" (UID: "0924a641-d0e7-4cdb-8130-f17a47a4c026") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:42.945588 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:42.945500    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgp4k\" (UniqueName: \"kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k\") pod \"network-check-target-6kpt2\" (UID: \"b7f9ded5-2460-4131-ab22-85c462aaae48\") " pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:42.945730 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:42.945678    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:42.945730 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:42.945695    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:42.945730 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:42.945708    2575 projected.go:196] Error preparing data for projected volume kube-api-access-hgp4k for pod openshift-network-diagnostics/network-check-target-6kpt2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:42.945838 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:42.945765    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k podName:b7f9ded5-2460-4131-ab22-85c462aaae48 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:50.945748102 +0000 UTC m=+17.520478643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hgp4k" (UniqueName: "kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k") pod "network-check-target-6kpt2" (UID: "b7f9ded5-2460-4131-ab22-85c462aaae48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:42.995639 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:42.995600    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:42.995822 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:42.995600    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:42.995822 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:42.995735    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbcj2" podUID="0924a641-d0e7-4cdb-8130-f17a47a4c026"
Nov 04 03:16:42.995822 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:42.995800    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lgvq4" podUID="23b77dc0-f7c4-4e72-a022-d75d8c8221fd"
Nov 04 03:16:42.996020 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:42.995876    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:42.996020 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:42.995969    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kpt2" podUID="b7f9ded5-2460-4131-ab22-85c462aaae48"
Nov 04 03:16:43.094941 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:43.094904    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4" event={"ID":"4db0986b-dec5-4a04-b0f0-ebbc3298f250","Type":"ContainerStarted","Data":"b1f44e046052771faeacd38f65cf0119a66f3630a49b318ed6d081f3b103ff08"}
Nov 04 03:16:43.097698 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:43.097668    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tdgzf" event={"ID":"18329af9-cec7-48fd-aeee-18367bb6c078","Type":"ContainerStarted","Data":"900a6d7e7a828819cae7248aec2bba332ecd1c5be939d66b8d0cbb509b69c3ed"}
Nov 04 03:16:44.099776 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:44.099180    2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 03:16:44.282813 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:44.282669    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-sd8pw"
Nov 04 03:16:44.283477 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:44.283270    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sd8pw"
Nov 04 03:16:44.401685 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:44.401591    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tdgzf" podStartSLOduration=4.707816391 podStartE2EDuration="10.40157096s" totalImagesPullingTime="5.693754569s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:43.215614324 +0000 UTC m=+9.790344885" watchObservedRunningTime="2026-11-04 03:16:44.40157096 +0000 UTC m=+10.976301539"
Nov 04 03:16:44.996492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:44.995937    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:44.996492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:44.995952    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:44.996492 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:44.995952    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:44.996492 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:44.996085    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lgvq4" podUID="23b77dc0-f7c4-4e72-a022-d75d8c8221fd"
Nov 04 03:16:44.996492 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:44.996207    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kpt2" podUID="b7f9ded5-2460-4131-ab22-85c462aaae48"
Nov 04 03:16:44.996492 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:44.996290    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbcj2" podUID="0924a641-d0e7-4cdb-8130-f17a47a4c026"
Nov 04 03:16:45.110001 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:45.109923    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hwrqp" event={"ID":"7721e70f-2dda-400b-b1a7-d434249efb7e","Type":"ContainerStarted","Data":"29fbe0f94af610ce3e365afacbed8592f7309e18237e78751193ce3e94d176ce"}
Nov 04 03:16:45.112799 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:45.112763    2575 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 03:16:45.231149 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:45.231101    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-hwrqp" podStartSLOduration=2.846109078 podStartE2EDuration="11.231086142s" totalImagesPullingTime="8.384977064s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:45.230513505 +0000 UTC m=+11.805244067" watchObservedRunningTime="2026-11-04 03:16:45.231086142 +0000 UTC m=+11.805816704"
Nov 04 03:16:45.930634 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:45.929866    2575 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T03:16:45.112785842Z","UUID":"418f1ad0-891f-4a89-ab98-329e928ef98b","Handler":null,"Name":"","Endpoint":""}
Nov 04 03:16:45.931986 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:45.931960    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 03:16:45.932103 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:45.931997    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 03:16:46.114380 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:46.114268    2575 generic.go:356] "Generic (PLEG): container finished" podID="ea8cb5af-3757-47db-840e-728d8d0ad62d" containerID="15220f82bf17151512fdfdc57e4cf1cca778350a143c80410bbdf3f3528a46ea" exitCode=0
Nov 04 03:16:46.115047 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:46.114382    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrrqq" event={"ID":"ea8cb5af-3757-47db-840e-728d8d0ad62d","Type":"ContainerDied","Data":"15220f82bf17151512fdfdc57e4cf1cca778350a143c80410bbdf3f3528a46ea"}
Nov 04 03:16:46.116748 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:46.116718    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4" event={"ID":"4db0986b-dec5-4a04-b0f0-ebbc3298f250","Type":"ContainerStarted","Data":"9b60adaa66d5e4c9086de65b27d18ee7c14e035cd62e792388d51021e2f6d9bb"}
Nov 04 03:16:46.119323 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:46.119300    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qk4nr" event={"ID":"70ed447c-add1-4e78-94fb-6aa3e4389fb5","Type":"ContainerStarted","Data":"d0d313975d0796a068d442b082d3caa1efeb0b79595e622eccf82a6af84a998e"}
Nov 04 03:16:46.357686 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:46.357622    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-qk4nr" podStartSLOduration=3.593837429 podStartE2EDuration="12.357603693s" totalImagesPullingTime="8.763766264s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:46.356731796 +0000 UTC m=+12.931462352" watchObservedRunningTime="2026-11-04 03:16:46.357603693 +0000 UTC m=+12.932334253"
Nov 04 03:16:46.995659 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:46.995462    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:46.995785 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:46.995765    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbcj2" podUID="0924a641-d0e7-4cdb-8130-f17a47a4c026"
Nov 04 03:16:46.995864 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:46.995518    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:46.995930 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:46.995518    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:46.995985 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:46.995952    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lgvq4" podUID="23b77dc0-f7c4-4e72-a022-d75d8c8221fd"
Nov 04 03:16:46.996087 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:46.996033    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kpt2" podUID="b7f9ded5-2460-4131-ab22-85c462aaae48"
Nov 04 03:16:47.125011 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:47.124979    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4" event={"ID":"4db0986b-dec5-4a04-b0f0-ebbc3298f250","Type":"ContainerStarted","Data":"cf7d8932d0935bf5a4ad8226dad86d8d17a5d6ce30f32dd518638050c0d4e858"}
Nov 04 03:16:47.127245 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:47.127216    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bkcnt" event={"ID":"16976140-0bb0-4983-bcb7-b2fa2274d24e","Type":"ContainerStarted","Data":"7b821d321949804b58729503b88af9103fa1a5462a793d7aa67b616092fb5a41"}
Nov 04 03:16:47.130017 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:47.129993    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m9zlr" event={"ID":"5d04c72f-9d65-429a-ad6c-dc69054ba75e","Type":"ContainerStarted","Data":"2ada485744bc1e6edd019d1a9cff703dd4544795b787870300c0a59058327aab"}
Nov 04 03:16:47.246246 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:47.246149    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4" podStartSLOduration=3.08732164 podStartE2EDuration="13.246134239s" totalImagesPullingTime="10.158812599s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:47.246120397 +0000 UTC m=+13.820850999" watchObservedRunningTime="2026-11-04 03:16:47.246134239 +0000 UTC m=+13.820864792"
Nov 04 03:16:47.487483 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:47.487426    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m9zlr" podStartSLOduration=5.041041569 podStartE2EDuration="13.487408106s" totalImagesPullingTime="8.446366537s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:47.487054451 +0000 UTC m=+14.061785011" watchObservedRunningTime="2026-11-04 03:16:47.487408106 +0000 UTC m=+14.062138694"
Nov 04 03:16:47.487770 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:47.487744    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bkcnt" podStartSLOduration=2.962143765 podStartE2EDuration="13.487734891s" totalImagesPullingTime="10.525591126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:47.366624327 +0000 UTC m=+13.941354888" watchObservedRunningTime="2026-11-04 03:16:47.487734891 +0000 UTC m=+14.062465452"
Nov 04 03:16:48.134276 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:48.134241    2575 generic.go:356] "Generic (PLEG): container finished" podID="ea8cb5af-3757-47db-840e-728d8d0ad62d" containerID="6847ebc17c8471c06fbebfc5a2d1164540b654305a9944baf17c479c155e4d74" exitCode=0
Nov 04 03:16:48.134969 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:48.134362    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrrqq" event={"ID":"ea8cb5af-3757-47db-840e-728d8d0ad62d","Type":"ContainerDied","Data":"6847ebc17c8471c06fbebfc5a2d1164540b654305a9944baf17c479c155e4d74"}
Nov 04 03:16:48.995322 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:48.995291    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:48.995506 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:48.995291    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:48.995506 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:48.995422    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kpt2" podUID="b7f9ded5-2460-4131-ab22-85c462aaae48"
Nov 04 03:16:48.995506 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:48.995298    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:48.995630 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:48.995507    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbcj2" podUID="0924a641-d0e7-4cdb-8130-f17a47a4c026"
Nov 04 03:16:48.995630 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:48.995548    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lgvq4" podUID="23b77dc0-f7c4-4e72-a022-d75d8c8221fd"
Nov 04 03:16:50.141690 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:50.141655    2575 generic.go:356] "Generic (PLEG): container finished" podID="ea8cb5af-3757-47db-840e-728d8d0ad62d" containerID="812171ba9eafc8aa7f8bf650c8044f746306b01ac107b9634189c72396750aed" exitCode=0
Nov 04 03:16:50.142309 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:50.141712    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrrqq" event={"ID":"ea8cb5af-3757-47db-840e-728d8d0ad62d","Type":"ContainerDied","Data":"812171ba9eafc8aa7f8bf650c8044f746306b01ac107b9634189c72396750aed"}
Nov 04 03:16:50.908397 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:50.908215    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:50.908566 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:50.908407    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs\") pod \"network-metrics-daemon-lgvq4\" (UID: \"23b77dc0-f7c4-4e72-a022-d75d8c8221fd\") " pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:50.908566 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:50.908361    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:50.908566 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:50.908516    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret podName:0924a641-d0e7-4cdb-8130-f17a47a4c026 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:06.908499873 +0000 UTC m=+33.483230411 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret") pod "global-pull-secret-syncer-qbcj2" (UID: "0924a641-d0e7-4cdb-8130-f17a47a4c026") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:50.908566 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:50.908518    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:50.908566 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:50.908566    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs podName:23b77dc0-f7c4-4e72-a022-d75d8c8221fd nodeName:}" failed. No retries permitted until 2026-11-04 03:17:06.908551739 +0000 UTC m=+33.483282282 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs") pod "network-metrics-daemon-lgvq4" (UID: "23b77dc0-f7c4-4e72-a022-d75d8c8221fd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:50.995822 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:50.995802    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:50.995912 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:50.995880    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:50.995983 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:50.995968    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:50.996025 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:50.995971    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kpt2" podUID="b7f9ded5-2460-4131-ab22-85c462aaae48"
Nov 04 03:16:50.996062 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:50.996042    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lgvq4" podUID="23b77dc0-f7c4-4e72-a022-d75d8c8221fd"
Nov 04 03:16:50.996151 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:50.996126    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbcj2" podUID="0924a641-d0e7-4cdb-8130-f17a47a4c026"
Nov 04 03:16:51.009407 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:51.009389    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgp4k\" (UniqueName: \"kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k\") pod \"network-check-target-6kpt2\" (UID: \"b7f9ded5-2460-4131-ab22-85c462aaae48\") " pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:51.009500 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:51.009489    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:51.009544 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:51.009505    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:51.009544 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:51.009523    2575 projected.go:196] Error preparing data for projected volume kube-api-access-hgp4k for pod openshift-network-diagnostics/network-check-target-6kpt2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:51.009625 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:51.009560    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k podName:b7f9ded5-2460-4131-ab22-85c462aaae48 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:07.009550372 +0000 UTC m=+33.584280923 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hgp4k" (UniqueName: "kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k") pod "network-check-target-6kpt2" (UID: "b7f9ded5-2460-4131-ab22-85c462aaae48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:51.145616 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:51.145586    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5" event={"ID":"a4fa5e68-1a28-41d2-9680-c79f0d95e361","Type":"ContainerStarted","Data":"232b2e5f60511865d292d38a94cd0307ad9763260da8900078a9c05127189776"}
Nov 04 03:16:51.145895 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:51.145651    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5" event={"ID":"a4fa5e68-1a28-41d2-9680-c79f0d95e361","Type":"ContainerStarted","Data":"8e6bc5925e842013f1d4a50ee1c205414e4d9e60782348ac3f8a7f2505011ca1"}
Nov 04 03:16:51.145895 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:51.145679    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5" event={"ID":"a4fa5e68-1a28-41d2-9680-c79f0d95e361","Type":"ContainerStarted","Data":"14fa755df0b2a046b462a957401a4e15d0dbc2b4d56b0f12689a18687047bc74"}
Nov 04 03:16:52.152258 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:52.152054    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5" event={"ID":"a4fa5e68-1a28-41d2-9680-c79f0d95e361","Type":"ContainerStarted","Data":"caf0d2caaaecb22356f2282f8b300e355733f941e968b80e61bbd9225e331bc9"}
Nov 04 03:16:52.152736 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:52.152311    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5" event={"ID":"a4fa5e68-1a28-41d2-9680-c79f0d95e361","Type":"ContainerStarted","Data":"c883133d7c7f722b0f847ae809b431a42566315591465b8d6984b4d9a6ab5e7f"}
Nov 04 03:16:52.152736 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:52.152344    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5" event={"ID":"a4fa5e68-1a28-41d2-9680-c79f0d95e361","Type":"ContainerStarted","Data":"0e64d13d77e41b0bd58ec2e602401db665a5fb1f934e5b50dc3d359dd35495a9"}
Nov 04 03:16:52.994981 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:52.994950    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:52.995155 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:52.995010    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:52.995155 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:52.995098    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:52.995155 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:52.995103    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kpt2" podUID="b7f9ded5-2460-4131-ab22-85c462aaae48"
Nov 04 03:16:52.995269 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:52.995159    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lgvq4" podUID="23b77dc0-f7c4-4e72-a022-d75d8c8221fd"
Nov 04 03:16:52.995269 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:52.995218    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbcj2" podUID="0924a641-d0e7-4cdb-8130-f17a47a4c026"
Nov 04 03:16:53.156304 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:53.156273    2575 generic.go:356] "Generic (PLEG): container finished" podID="ea8cb5af-3757-47db-840e-728d8d0ad62d" containerID="40d3626a79217c6e2edc4ab867e1c261d83cba0ccbc2ed8d50e584d1df1ee6ea" exitCode=0
Nov 04 03:16:53.156783 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:53.156319    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrrqq" event={"ID":"ea8cb5af-3757-47db-840e-728d8d0ad62d","Type":"ContainerDied","Data":"40d3626a79217c6e2edc4ab867e1c261d83cba0ccbc2ed8d50e584d1df1ee6ea"}
Nov 04 03:16:54.161240 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:54.161201    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5" event={"ID":"a4fa5e68-1a28-41d2-9680-c79f0d95e361","Type":"ContainerStarted","Data":"11b091094aa14e7c19a64be7e91f4353ae3f837b7db382c4efc27358b9769ebb"}
Nov 04 03:16:54.164044 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:54.164017    2575 generic.go:356] "Generic (PLEG): container finished" podID="ea8cb5af-3757-47db-840e-728d8d0ad62d" containerID="3d7c696a092faa900bc031134b69fe1d04f206db5b62511324753d9e5604bdb4" exitCode=0
Nov 04 03:16:54.164159 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:54.164058    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrrqq" event={"ID":"ea8cb5af-3757-47db-840e-728d8d0ad62d","Type":"ContainerDied","Data":"3d7c696a092faa900bc031134b69fe1d04f206db5b62511324753d9e5604bdb4"}
Nov 04 03:16:54.995932 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:54.995896    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:54.995932 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:54.995928    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:54.996130 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:54.995896    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:54.996130 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:54.996011    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbcj2" podUID="0924a641-d0e7-4cdb-8130-f17a47a4c026"
Nov 04 03:16:54.996130 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:54.996066    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kpt2" podUID="b7f9ded5-2460-4131-ab22-85c462aaae48"
Nov 04 03:16:54.996224 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:54.996150    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lgvq4" podUID="23b77dc0-f7c4-4e72-a022-d75d8c8221fd"
Nov 04 03:16:55.169784 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:55.169742    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrrqq" event={"ID":"ea8cb5af-3757-47db-840e-728d8d0ad62d","Type":"ContainerStarted","Data":"e20879327719d68b54c470fe93872beae601bfaae8ac6270d2936e86aaca1890"}
Nov 04 03:16:55.294960 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:55.294913    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xrrqq" podStartSLOduration=8.538609891 podStartE2EDuration="21.294899241s" totalImagesPullingTime="12.75628935s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:55.294734565 +0000 UTC m=+21.869465127" watchObservedRunningTime="2026-11-04 03:16:55.294899241 +0000 UTC m=+21.869629799"
Nov 04 03:16:56.175753 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:56.175583    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5" event={"ID":"a4fa5e68-1a28-41d2-9680-c79f0d95e361","Type":"ContainerStarted","Data":"c2de6998765c5356a66aa5cd78c93c58f66e84ee3d5bf7c5ab8bdbd46c9870c9"}
Nov 04 03:16:56.450931 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:56.450829    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5" podStartSLOduration=7.965456074 podStartE2EDuration="22.450816854s" totalImagesPullingTime="14.48536078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:56.450693499 +0000 UTC m=+23.025424058" watchObservedRunningTime="2026-11-04 03:16:56.450816854 +0000 UTC m=+23.025547414"
Nov 04 03:16:56.995853 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:56.995803    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:56.996011 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:56.995894    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:56.996011 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:56.995989    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:56.996011 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:56.995998    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kpt2" podUID="b7f9ded5-2460-4131-ab22-85c462aaae48"
Nov 04 03:16:56.996121 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:56.996074    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbcj2" podUID="0924a641-d0e7-4cdb-8130-f17a47a4c026"
Nov 04 03:16:56.996163 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:56.996137    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lgvq4" podUID="23b77dc0-f7c4-4e72-a022-d75d8c8221fd"
Nov 04 03:16:57.177618 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:57.177593    2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 03:16:57.178155 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:57.177948    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:57.178155 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:57.178029    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:57.194481 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:57.194450    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:57.194772 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:57.194758    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:16:58.179918 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:58.179835    2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 03:16:58.338757 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:58.338698    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6kpt2"]
Nov 04 03:16:58.338977 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:58.338858    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:58.339055 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:58.339020    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kpt2" podUID="b7f9ded5-2460-4131-ab22-85c462aaae48"
Nov 04 03:16:58.339423 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:58.339399    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qbcj2"]
Nov 04 03:16:58.339531 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:58.339471    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:58.339568 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:58.339534    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbcj2" podUID="0924a641-d0e7-4cdb-8130-f17a47a4c026"
Nov 04 03:16:58.340763 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:58.340745    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lgvq4"]
Nov 04 03:16:58.340864 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:58.340813    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:58.340922 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:58.340906    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lgvq4" podUID="23b77dc0-f7c4-4e72-a022-d75d8c8221fd"
Nov 04 03:16:59.181326 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:59.181296    2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 03:16:59.995648 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:59.995567    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:16:59.995648 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:59.995613    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:16:59.995648 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:16:59.995627    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:16:59.995909 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:59.995694    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kpt2" podUID="b7f9ded5-2460-4131-ab22-85c462aaae48"
Nov 04 03:16:59.995909 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:59.995755    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbcj2" podUID="0924a641-d0e7-4cdb-8130-f17a47a4c026"
Nov 04 03:16:59.995909 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:16:59.995867    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lgvq4" podUID="23b77dc0-f7c4-4e72-a022-d75d8c8221fd"
Nov 04 03:17:01.995897 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:01.995780    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:17:01.995897 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:01.995780    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:17:01.996313 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:01.995923    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lgvq4" podUID="23b77dc0-f7c4-4e72-a022-d75d8c8221fd"
Nov 04 03:17:01.996313 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:01.995783    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:17:01.996313 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:01.995987    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbcj2" podUID="0924a641-d0e7-4cdb-8130-f17a47a4c026"
Nov 04 03:17:01.996313 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:01.996032    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kpt2" podUID="b7f9ded5-2460-4131-ab22-85c462aaae48"
Nov 04 03:17:03.365503 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.365470    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:03.365913 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.365669    2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 03:17:03.385320 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.385100    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:03.632654 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.632596    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeReady"
Nov 04 03:17:03.632776 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.632734    2575 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 04 03:17:03.727559 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.727532    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vwf9m"]
Nov 04 03:17:03.734054 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.734032    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xhrpk"]
Nov 04 03:17:03.734181 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.734142    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:03.742149 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.742133    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xhrpk"
Nov 04 03:17:03.789225 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.789199    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 03:17:03.789415 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.789257    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 03:17:03.789756 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.789729    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vjndb\""
Nov 04 03:17:03.790317 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.790298    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vwf9m"]
Nov 04 03:17:03.795065 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.795043    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:03.795162 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.795080    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86ljh\" (UniqueName: \"kubernetes.io/projected/a47538cd-f8b8-409f-8eed-c0d9881a822e-kube-api-access-86ljh\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:03.795162 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.795102    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae6f9681-26d2-4340-8b43-fd6092388459-cert\") pod \"ingress-canary-xhrpk\" (UID: \"ae6f9681-26d2-4340-8b43-fd6092388459\") " pod="openshift-ingress-canary/ingress-canary-xhrpk"
Nov 04 03:17:03.795287 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.795264    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a47538cd-f8b8-409f-8eed-c0d9881a822e-tmp-dir\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:03.795331 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.795321    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvrhn\" (UniqueName: \"kubernetes.io/projected/ae6f9681-26d2-4340-8b43-fd6092388459-kube-api-access-jvrhn\") pod \"ingress-canary-xhrpk\" (UID: \"ae6f9681-26d2-4340-8b43-fd6092388459\") " pod="openshift-ingress-canary/ingress-canary-xhrpk"
Nov 04 03:17:03.795377 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.795340    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a47538cd-f8b8-409f-8eed-c0d9881a822e-config-volume\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:03.796091 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.796073    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 03:17:03.796172 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.796105    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 03:17:03.796172 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.796129    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-mhdb8\""
Nov 04 03:17:03.796172 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.796142    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 03:17:03.811581 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.811550    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xhrpk"]
Nov 04 03:17:03.896312 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.896244    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae6f9681-26d2-4340-8b43-fd6092388459-cert\") pod \"ingress-canary-xhrpk\" (UID: \"ae6f9681-26d2-4340-8b43-fd6092388459\") " pod="openshift-ingress-canary/ingress-canary-xhrpk"
Nov 04 03:17:03.896312 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.896283    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a47538cd-f8b8-409f-8eed-c0d9881a822e-tmp-dir\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:03.896312 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.896303    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvrhn\" (UniqueName: \"kubernetes.io/projected/ae6f9681-26d2-4340-8b43-fd6092388459-kube-api-access-jvrhn\") pod \"ingress-canary-xhrpk\" (UID: \"ae6f9681-26d2-4340-8b43-fd6092388459\") " pod="openshift-ingress-canary/ingress-canary-xhrpk"
Nov 04 03:17:03.896548 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.896321    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a47538cd-f8b8-409f-8eed-c0d9881a822e-config-volume\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:03.896548 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.896372    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:03.896548 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.896395    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86ljh\" (UniqueName: \"kubernetes.io/projected/a47538cd-f8b8-409f-8eed-c0d9881a822e-kube-api-access-86ljh\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:03.896548 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:03.896505    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:17:03.896742 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:03.896575    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls podName:a47538cd-f8b8-409f-8eed-c0d9881a822e nodeName:}" failed. No retries permitted until 2026-11-04 03:17:04.396556102 +0000 UTC m=+30.971286654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls") pod "dns-default-vwf9m" (UID: "a47538cd-f8b8-409f-8eed-c0d9881a822e") : secret "dns-default-metrics-tls" not found
Nov 04 03:17:03.896742 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.896688    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a47538cd-f8b8-409f-8eed-c0d9881a822e-tmp-dir\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:03.897029 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.897011    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a47538cd-f8b8-409f-8eed-c0d9881a822e-config-volume\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:03.900888 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.900869    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae6f9681-26d2-4340-8b43-fd6092388459-cert\") pod \"ingress-canary-xhrpk\" (UID: \"ae6f9681-26d2-4340-8b43-fd6092388459\") " pod="openshift-ingress-canary/ingress-canary-xhrpk"
Nov 04 03:17:03.959960 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.959882    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86ljh\" (UniqueName: \"kubernetes.io/projected/a47538cd-f8b8-409f-8eed-c0d9881a822e-kube-api-access-86ljh\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:03.960124 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.960097    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvrhn\" (UniqueName: \"kubernetes.io/projected/ae6f9681-26d2-4340-8b43-fd6092388459-kube-api-access-jvrhn\") pod \"ingress-canary-xhrpk\" (UID: \"ae6f9681-26d2-4340-8b43-fd6092388459\") " pod="openshift-ingress-canary/ingress-canary-xhrpk"
Nov 04 03:17:03.996479 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.996450    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:17:03.996627 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.996568    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:17:03.996863 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:03.996752    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:17:04.051373 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:04.051342    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 03:17:04.051508 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:04.051436    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 03:17:04.051761 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:04.051738    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 03:17:04.051838 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:04.051822    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xwzp4\""
Nov 04 03:17:04.052001 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:04.051984    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 03:17:04.052138 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:04.052000    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-zsv6g\""
Nov 04 03:17:04.052220 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:04.052138    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-mnqmt\""
Nov 04 03:17:04.076077 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:04.076052    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xhrpk"
Nov 04 03:17:04.400491 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:04.400446    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:04.401103 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:04.400566    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:17:04.401103 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:04.400649    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls podName:a47538cd-f8b8-409f-8eed-c0d9881a822e nodeName:}" failed. No retries permitted until 2026-11-04 03:17:05.400637204 +0000 UTC m=+31.975367742 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls") pod "dns-default-vwf9m" (UID: "a47538cd-f8b8-409f-8eed-c0d9881a822e") : secret "dns-default-metrics-tls" not found
Nov 04 03:17:04.485950 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:04.485927    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xhrpk"]
Nov 04 03:17:04.489326 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:17:04.489291    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae6f9681_26d2_4340_8b43_fd6092388459.slice/crio-6d864815b0dbcde352c4c05c2fd96425f4831e85b485d428c78be8e447d07fbc WatchSource:0}: Error finding container 6d864815b0dbcde352c4c05c2fd96425f4831e85b485d428c78be8e447d07fbc: Status 404 returned error can't find the container with id 6d864815b0dbcde352c4c05c2fd96425f4831e85b485d428c78be8e447d07fbc
Nov 04 03:17:05.195626 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.195595    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xhrpk" event={"ID":"ae6f9681-26d2-4340-8b43-fd6092388459","Type":"ContainerStarted","Data":"6d864815b0dbcde352c4c05c2fd96425f4831e85b485d428c78be8e447d07fbc"}
Nov 04 03:17:05.408770 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.408733    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:05.409141 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:05.408901    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:17:05.409141 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:05.408966    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls podName:a47538cd-f8b8-409f-8eed-c0d9881a822e nodeName:}" failed. No retries permitted until 2026-11-04 03:17:07.408950905 +0000 UTC m=+33.983681444 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls") pod "dns-default-vwf9m" (UID: "a47538cd-f8b8-409f-8eed-c0d9881a822e") : secret "dns-default-metrics-tls" not found
Nov 04 03:17:05.727943 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.727910    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-t5zsk"]
Nov 04 03:17:05.730595 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.730564    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:05.785222 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.785175    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 03:17:05.785394 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.785219    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 03:17:05.785394 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.785266    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 03:17:05.785394 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.785310    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 03:17:05.785394 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.785374    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jw2mq\""
Nov 04 03:17:05.791321 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.791297    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t5zsk"]
Nov 04 03:17:05.812363 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.812341    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/41f71514-1213-429b-8103-5fb88c9c7039-data-volume\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:05.812491 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.812406    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/41f71514-1213-429b-8103-5fb88c9c7039-crio-socket\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:05.812491 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.812469    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:05.812569 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.812492    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:05.812569 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.812511    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jwdt\" (UniqueName: \"kubernetes.io/projected/41f71514-1213-429b-8103-5fb88c9c7039-kube-api-access-7jwdt\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:05.913194 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.913146    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/41f71514-1213-429b-8103-5fb88c9c7039-data-volume\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:05.913365 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.913263    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/41f71514-1213-429b-8103-5fb88c9c7039-crio-socket\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:05.913365 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.913292    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:05.913365 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.913309    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:05.913365 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.913335    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jwdt\" (UniqueName: \"kubernetes.io/projected/41f71514-1213-429b-8103-5fb88c9c7039-kube-api-access-7jwdt\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:05.913557 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:05.913442    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:05.913557 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:05.913538    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:06.413516414 +0000 UTC m=+32.988246958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:05.913634 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.913558    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/41f71514-1213-429b-8103-5fb88c9c7039-data-volume\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:05.913670 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:05.913659    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:05.913714 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:05.913703    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:06.413693401 +0000 UTC m=+32.988423939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:05.913770 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.913752    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/41f71514-1213-429b-8103-5fb88c9c7039-crio-socket\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:05.979257 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.979183    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jwdt\" (UniqueName: \"kubernetes.io/projected/41f71514-1213-429b-8103-5fb88c9c7039-kube-api-access-7jwdt\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:05.987173 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.987148    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-dm5lv-xwxch"]
Nov 04 03:17:05.989649 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:05.989630    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:06.044148 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.044124    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 04 03:17:06.044669 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.044648    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-gj69c\""
Nov 04 03:17:06.114511 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.114476    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-service-ca-bundle\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:06.114664 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.114642    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmgmq\" (UniqueName: \"kubernetes.io/projected/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-kube-api-access-bmgmq\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:06.114719 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.114677    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-trusted-ca-bundle\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:06.114719 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.114707    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-archives-path\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:06.131247 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.131223    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 04 03:17:06.148757 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.148685    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-dm5lv-xwxch"]
Nov 04 03:17:06.215055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.215022    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmgmq\" (UniqueName: \"kubernetes.io/projected/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-kube-api-access-bmgmq\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:06.215055 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.215060    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-trusted-ca-bundle\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:06.215366 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.215086    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-archives-path\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:06.215366 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.215343    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-service-ca-bundle\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:06.215530 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.215507    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-archives-path\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:06.215900 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.215879    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-service-ca-bundle\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:06.216149 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.216125    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-trusted-ca-bundle\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:06.279690 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.279659    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmgmq\" (UniqueName: \"kubernetes.io/projected/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-kube-api-access-bmgmq\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:06.302704 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.302675    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:06.417528 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.417495    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:06.417999 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.417540    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:06.417999 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:06.417632    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:06.417999 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:06.417701    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:06.417999 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:06.417712    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:07.417695805 +0000 UTC m=+33.992426363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:06.417999 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:06.417753    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:07.417738119 +0000 UTC m=+33.992468662 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:06.678858 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.678771    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-dm5lv-xwxch"]
Nov 04 03:17:06.685743 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:17:06.685716    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97184dbe_af1d_408e_8dbc_0822eb2f2d8d.slice/crio-6e0704982824dc3fb0713a5fc64778a70fcd7b9ad8bad1b9aaf2dfcd530d119d WatchSource:0}: Error finding container 6e0704982824dc3fb0713a5fc64778a70fcd7b9ad8bad1b9aaf2dfcd530d119d: Status 404 returned error can't find the container with id 6e0704982824dc3fb0713a5fc64778a70fcd7b9ad8bad1b9aaf2dfcd530d119d
Nov 04 03:17:06.704180 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.704157    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-9fzvp"]
Nov 04 03:17:06.707488 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.707469    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9fzvp"
Nov 04 03:17:06.763756 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.763722    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 04 03:17:06.763917 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.763723    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 04 03:17:06.763917 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.763896    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5zf5b\""
Nov 04 03:17:06.764699 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.764678    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-9fzvp"]
Nov 04 03:17:06.821491 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.821461    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98vqg\" (UniqueName: \"kubernetes.io/projected/62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8-kube-api-access-98vqg\") pod \"migrator-66df5d496b-9fzvp\" (UID: \"62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9fzvp"
Nov 04 03:17:06.922374 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.922334    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:17:06.922555 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.922391    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs\") pod \"network-metrics-daemon-lgvq4\" (UID: \"23b77dc0-f7c4-4e72-a022-d75d8c8221fd\") " pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:17:06.922555 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:06.922507    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 03:17:06.922669 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:06.922566    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs podName:23b77dc0-f7c4-4e72-a022-d75d8c8221fd nodeName:}" failed. No retries permitted until 2026-11-04 03:17:38.922552287 +0000 UTC m=+65.497282825 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs") pod "network-metrics-daemon-lgvq4" (UID: "23b77dc0-f7c4-4e72-a022-d75d8c8221fd") : secret "metrics-daemon-secret" not found
Nov 04 03:17:06.922669 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.922599    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98vqg\" (UniqueName: \"kubernetes.io/projected/62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8-kube-api-access-98vqg\") pod \"migrator-66df5d496b-9fzvp\" (UID: \"62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9fzvp"
Nov 04 03:17:06.925188 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.925164    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:17:06.987688 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:06.987623    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98vqg\" (UniqueName: \"kubernetes.io/projected/62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8-kube-api-access-98vqg\") pod \"migrator-66df5d496b-9fzvp\" (UID: \"62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9fzvp"
Nov 04 03:17:07.018593 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.018553    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9fzvp"
Nov 04 03:17:07.024019 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.023984    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgp4k\" (UniqueName: \"kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k\") pod \"network-check-target-6kpt2\" (UID: \"b7f9ded5-2460-4131-ab22-85c462aaae48\") " pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:17:07.026935 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.026913    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgp4k\" (UniqueName: \"kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k\") pod \"network-check-target-6kpt2\" (UID: \"b7f9ded5-2460-4131-ab22-85c462aaae48\") " pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:17:07.038746 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.038723    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:17:07.042670 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.042645    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:17:07.201484 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.201448    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-dm5lv-xwxch" event={"ID":"97184dbe-af1d-408e-8dbc-0822eb2f2d8d","Type":"ContainerStarted","Data":"6e0704982824dc3fb0713a5fc64778a70fcd7b9ad8bad1b9aaf2dfcd530d119d"}
Nov 04 03:17:07.399425 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.399398    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-9fzvp"]
Nov 04 03:17:07.401078 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:17:07.401055    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b0b5f0_3fb2_4d16_9017_3ae27a0e84e8.slice/crio-984183d79b7c2b4c1084d38a28e1d1ffc701595e0a0026b51aacf9c4c62b0ff1 WatchSource:0}: Error finding container 984183d79b7c2b4c1084d38a28e1d1ffc701595e0a0026b51aacf9c4c62b0ff1: Status 404 returned error can't find the container with id 984183d79b7c2b4c1084d38a28e1d1ffc701595e0a0026b51aacf9c4c62b0ff1
Nov 04 03:17:07.428614 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.428583    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:07.429028 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.428667    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:07.429028 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.428695    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:07.429028 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:07.428749    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:07.429028 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:07.428767    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:07.429028 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:07.428825    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:09.428804562 +0000 UTC m=+36.003535116 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:07.429028 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:07.428830    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:17:07.429028 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:07.428857    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:09.428837889 +0000 UTC m=+36.003568428 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:07.429028 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:07.428912    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls podName:a47538cd-f8b8-409f-8eed-c0d9881a822e nodeName:}" failed. No retries permitted until 2026-11-04 03:17:11.428896313 +0000 UTC m=+38.003626852 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls") pod "dns-default-vwf9m" (UID: "a47538cd-f8b8-409f-8eed-c0d9881a822e") : secret "dns-default-metrics-tls" not found
Nov 04 03:17:07.449609 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.449588    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qbcj2"]
Nov 04 03:17:07.452201 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:17:07.452171    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0924a641_d0e7_4cdb_8130_f17a47a4c026.slice/crio-cec18679e27eb17487e79dfdf14c395c949dcdc91fd2f74f8e6d84f0942334a2 WatchSource:0}: Error finding container cec18679e27eb17487e79dfdf14c395c949dcdc91fd2f74f8e6d84f0942334a2: Status 404 returned error can't find the container with id cec18679e27eb17487e79dfdf14c395c949dcdc91fd2f74f8e6d84f0942334a2
Nov 04 03:17:07.452706 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.452665    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6kpt2"]
Nov 04 03:17:07.456197 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:17:07.456173    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7f9ded5_2460_4131_ab22_85c462aaae48.slice/crio-9c43cf7b698ca862824757a46893033b7767cc57d6863ac457874e47434de10b WatchSource:0}: Error finding container 9c43cf7b698ca862824757a46893033b7767cc57d6863ac457874e47434de10b: Status 404 returned error can't find the container with id 9c43cf7b698ca862824757a46893033b7767cc57d6863ac457874e47434de10b
Nov 04 03:17:07.697419 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.697396    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-pnj7z"]
Nov 04 03:17:07.700507 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.700488    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z"
Nov 04 03:17:07.758010 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.757964    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 04 03:17:07.758010 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.757991    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 04 03:17:07.758010 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.757971    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 04 03:17:07.758179 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.758110    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 04 03:17:07.758242 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.758192    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-xs24w\""
Nov 04 03:17:07.759298 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.759278    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-pnj7z"]
Nov 04 03:17:07.832312 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.832277    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ab11857e-7c79-464b-aba7-c6df7ef55a7f-signing-key\") pod \"service-ca-5fffc97bdf-pnj7z\" (UID: \"ab11857e-7c79-464b-aba7-c6df7ef55a7f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z"
Nov 04 03:17:07.832483 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.832366    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ab11857e-7c79-464b-aba7-c6df7ef55a7f-signing-cabundle\") pod \"service-ca-5fffc97bdf-pnj7z\" (UID: \"ab11857e-7c79-464b-aba7-c6df7ef55a7f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z"
Nov 04 03:17:07.832483 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.832415    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xts4\" (UniqueName: \"kubernetes.io/projected/ab11857e-7c79-464b-aba7-c6df7ef55a7f-kube-api-access-9xts4\") pod \"service-ca-5fffc97bdf-pnj7z\" (UID: \"ab11857e-7c79-464b-aba7-c6df7ef55a7f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z"
Nov 04 03:17:07.933932 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.933659    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ab11857e-7c79-464b-aba7-c6df7ef55a7f-signing-cabundle\") pod \"service-ca-5fffc97bdf-pnj7z\" (UID: \"ab11857e-7c79-464b-aba7-c6df7ef55a7f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z"
Nov 04 03:17:07.933932 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.933723    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xts4\" (UniqueName: \"kubernetes.io/projected/ab11857e-7c79-464b-aba7-c6df7ef55a7f-kube-api-access-9xts4\") pod \"service-ca-5fffc97bdf-pnj7z\" (UID: \"ab11857e-7c79-464b-aba7-c6df7ef55a7f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z"
Nov 04 03:17:07.933932 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.933793    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ab11857e-7c79-464b-aba7-c6df7ef55a7f-signing-key\") pod \"service-ca-5fffc97bdf-pnj7z\" (UID: \"ab11857e-7c79-464b-aba7-c6df7ef55a7f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z"
Nov 04 03:17:07.934420 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.934393    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ab11857e-7c79-464b-aba7-c6df7ef55a7f-signing-cabundle\") pod \"service-ca-5fffc97bdf-pnj7z\" (UID: \"ab11857e-7c79-464b-aba7-c6df7ef55a7f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z"
Nov 04 03:17:07.936478 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.936456    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ab11857e-7c79-464b-aba7-c6df7ef55a7f-signing-key\") pod \"service-ca-5fffc97bdf-pnj7z\" (UID: \"ab11857e-7c79-464b-aba7-c6df7ef55a7f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z"
Nov 04 03:17:07.997715 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:07.997687    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xts4\" (UniqueName: \"kubernetes.io/projected/ab11857e-7c79-464b-aba7-c6df7ef55a7f-kube-api-access-9xts4\") pod \"service-ca-5fffc97bdf-pnj7z\" (UID: \"ab11857e-7c79-464b-aba7-c6df7ef55a7f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z"
Nov 04 03:17:08.036186 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:08.036153    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z"
Nov 04 03:17:08.205170 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:08.205082    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9fzvp" event={"ID":"62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8","Type":"ContainerStarted","Data":"984183d79b7c2b4c1084d38a28e1d1ffc701595e0a0026b51aacf9c4c62b0ff1"}
Nov 04 03:17:08.206816 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:08.206787    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xhrpk" event={"ID":"ae6f9681-26d2-4340-8b43-fd6092388459","Type":"ContainerStarted","Data":"cccc364a9a0994cc8e383f4fd4cd3be8afe2cbeee1bcbe1f19e31047fbb888a8"}
Nov 04 03:17:08.208176 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:08.208146    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qbcj2" event={"ID":"0924a641-d0e7-4cdb-8130-f17a47a4c026","Type":"ContainerStarted","Data":"cec18679e27eb17487e79dfdf14c395c949dcdc91fd2f74f8e6d84f0942334a2"}
Nov 04 03:17:08.209517 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:08.209483    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6kpt2" event={"ID":"b7f9ded5-2460-4131-ab22-85c462aaae48","Type":"ContainerStarted","Data":"9c43cf7b698ca862824757a46893033b7767cc57d6863ac457874e47434de10b"}
Nov 04 03:17:08.411608 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:08.411558    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xhrpk" podStartSLOduration=2.229006199 podStartE2EDuration="5.411541785s" totalImagesPullingTime="3.182535586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:08.340736089 +0000 UTC m=+34.915466649" watchObservedRunningTime="2026-11-04 03:17:08.411541785 +0000 UTC m=+34.986272344"
Nov 04 03:17:08.412108 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:08.412088    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-pnj7z"]
Nov 04 03:17:08.413935 ip-10-0-135-242 kubenswrapper[2575]: W1104 03:17:08.413892    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab11857e_7c79_464b_aba7_c6df7ef55a7f.slice/crio-ee15f239bf435ca90f74e18f3ab023f838325097e419ce753d62a5c1216ce8b2 WatchSource:0}: Error finding container ee15f239bf435ca90f74e18f3ab023f838325097e419ce753d62a5c1216ce8b2: Status 404 returned error can't find the container with id ee15f239bf435ca90f74e18f3ab023f838325097e419ce753d62a5c1216ce8b2
Nov 04 03:17:09.219690 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:09.219572    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z" event={"ID":"ab11857e-7c79-464b-aba7-c6df7ef55a7f","Type":"ContainerStarted","Data":"ee15f239bf435ca90f74e18f3ab023f838325097e419ce753d62a5c1216ce8b2"}
Nov 04 03:17:09.448597 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:09.448559    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:09.448597 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:09.448600    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:09.448832 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:09.448724    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:09.448832 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:09.448786    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:13.448767431 +0000 UTC m=+40.023497970 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:09.448986 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:09.448859    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:09.448986 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:09.448896    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:13.448883638 +0000 UTC m=+40.023614191 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:10.224025 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:10.223965    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-dm5lv-xwxch" event={"ID":"97184dbe-af1d-408e-8dbc-0822eb2f2d8d","Type":"ContainerStarted","Data":"73e9e632fe2a67086b6eb08e1342e7886c729089d953f698fc398261e8fc080c"}
Nov 04 03:17:10.227306 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:10.227273    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9fzvp" event={"ID":"62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8","Type":"ContainerStarted","Data":"cb0407c85267f50f9e8fbc5d0c2a9f00af7d1814eeae49feb94c3b9268d5c37a"}
Nov 04 03:17:10.345126 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:10.344124    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-dm5lv-xwxch" podStartSLOduration=2.45382288 podStartE2EDuration="5.3441044s" totalImagesPullingTime="2.89028152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:10.343960628 +0000 UTC m=+36.918691190" watchObservedRunningTime="2026-11-04 03:17:10.3441044 +0000 UTC m=+36.918834963"
Nov 04 03:17:10.463673 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:10.463617    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9fzvp" podStartSLOduration=2.552591024 podStartE2EDuration="4.463595063s" totalImagesPullingTime="1.911004039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:10.462654014 +0000 UTC m=+37.037384573" watchObservedRunningTime="2026-11-04 03:17:10.463595063 +0000 UTC m=+37.038325623"
Nov 04 03:17:11.466609 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:11.466568    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:11.467085 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:11.466717    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:17:11.467085 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:11.466764    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls podName:a47538cd-f8b8-409f-8eed-c0d9881a822e nodeName:}" failed. No retries permitted until 2026-11-04 03:17:19.466751492 +0000 UTC m=+46.041482043 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls") pod "dns-default-vwf9m" (UID: "a47538cd-f8b8-409f-8eed-c0d9881a822e") : secret "dns-default-metrics-tls" not found
Nov 04 03:17:12.239344 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:12.239301    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6kpt2" event={"ID":"b7f9ded5-2460-4131-ab22-85c462aaae48","Type":"ContainerStarted","Data":"3c2f9997f1e5a4e1261e72841d08b82f3d1978defd4ea8bc3680dcfc342c7b78"}
Nov 04 03:17:12.239538 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:12.239400    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:17:12.241182 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:12.241144    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z" event={"ID":"ab11857e-7c79-464b-aba7-c6df7ef55a7f","Type":"ContainerStarted","Data":"4f60bdc72282639a9382ed29a0ab077b30376593b7a3c5b2c85682a082a9d2ff"}
Nov 04 03:17:12.358495 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:12.358412    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6kpt2" podStartSLOduration=34.123527512 podStartE2EDuration="38.358397942s" totalImagesPullingTime="4.23487043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:16:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:12.357676898 +0000 UTC m=+38.932407458" watchObservedRunningTime="2026-11-04 03:17:12.358397942 +0000 UTC m=+38.933128495"
Nov 04 03:17:12.477074 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:12.477008    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z" podStartSLOduration=2.048497561 podStartE2EDuration="5.476993041s" totalImagesPullingTime="3.42849548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:12.476280458 +0000 UTC m=+39.051011018" watchObservedRunningTime="2026-11-04 03:17:12.476993041 +0000 UTC m=+39.051723601"
Nov 04 03:17:13.482512 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:13.482464    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:13.482512 ip-10-0-135-242 kubenswrapper[2575]: I1104 03:17:13.482514    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:13.483095 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:13.482599    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:13.483095 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:13.482620    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:13.483095 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:13.482685    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:21.482665345 +0000 UTC m=+48.057395897 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:13.483095 ip-10-0-135-242 kubenswrapper[2575]: E1104 03:17:13.482701    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:21.482694767 +0000 UTC m=+48.057425305 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:13.731557 ip-10-0-135-242 systemd[1]: Stopping Kubernetes Kubelet...
Nov 04 03:17:13.749383 ip-10-0-135-242 systemd[1]: kubelet.service: Deactivated successfully.
Nov 04 03:17:13.749626 ip-10-0-135-242 systemd[1]: Stopped Kubernetes Kubelet.
Nov 04 03:17:13.749711 ip-10-0-135-242 systemd[1]: kubelet.service: Consumed 2.411s CPU time.
Nov 04 03:17:13.765351 ip-10-0-135-242 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 03:17:13.829724 ip-10-0-135-242 kubenswrapper[4521]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:17:13.829724 ip-10-0-135-242 kubenswrapper[4521]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 03:17:13.829724 ip-10-0-135-242 kubenswrapper[4521]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:17:13.829724 ip-10-0-135-242 kubenswrapper[4521]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:17:13.831903 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831805    4521 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:13.831903 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831900    4521 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:13.831903 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831905    4521 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:13.831903 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831908    4521 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831912    4521 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831915    4521 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831917    4521 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831925    4521 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831928    4521 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831931    4521 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831933    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831937    4521 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831942    4521 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831945    4521 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831947    4521 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831950    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831952    4521 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831955    4521 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831958    4521 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831960    4521 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831963    4521 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831965    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:13.832061 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831968    4521 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831971    4521 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831973    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831976    4521 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831979    4521 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831982    4521 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831984    4521 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831987    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831989    4521 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831992    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831994    4521 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831997    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.831999    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832002    4521 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832006    4521 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832009    4521 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832012    4521 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832015    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832017    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:13.832575 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832020    4521 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832023    4521 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832025    4521 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832028    4521 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832031    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832033    4521 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832036    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832038    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832041    4521 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832044    4521 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832046    4521 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832049    4521 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832051    4521 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832054    4521 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832056    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832060    4521 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832063    4521 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832065    4521 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832068    4521 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:13.833066 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832071    4521 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832073    4521 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832077    4521 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832079    4521 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832082    4521 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832084    4521 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832087    4521 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832090    4521 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832092    4521 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832095    4521 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832098    4521 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832101    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832104    4521 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832107    4521 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832109    4521 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832112    4521 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832115    4521 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832118    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832121    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832123    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:13.833538 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832126    4521 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832128    4521 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832131    4521 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832133    4521 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832136    4521 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832138    4521 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832141    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832144    4521 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832146    4521 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832149    4521 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832151    4521 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832154    4521 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832156    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832159    4521 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832162    4521 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832165    4521 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832167    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832170    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832172    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832175    4521 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832178    4521 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:13.834060 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832181    4521 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832184    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832188    4521 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832192    4521 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.832196    4521 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833001    4521 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833010    4521 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833014    4521 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833019    4521 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833023    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833026    4521 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833029    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833032    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833035    4521 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833037    4521 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833040    4521 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833042    4521 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833045    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833048    4521 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:13.834588 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833050    4521 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833053    4521 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833055    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833058    4521 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833061    4521 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833063    4521 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833067    4521 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833070    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833073    4521 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833075    4521 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833078    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833081    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833084    4521 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833087    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833090    4521 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833093    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833096    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833099    4521 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833102    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:13.835070 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833105    4521 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833108    4521 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833111    4521 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833113    4521 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833116    4521 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833118    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833121    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833124    4521 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833126    4521 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833129    4521 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833132    4521 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833134    4521 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833137    4521 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833141    4521 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833144    4521 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833148    4521 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833151    4521 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833154    4521 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833156    4521 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:13.835620 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833159    4521 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833162    4521 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833165    4521 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833168    4521 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833171    4521 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833174    4521 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833176    4521 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833179    4521 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833182    4521 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833184    4521 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833187    4521 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833189    4521 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833192    4521 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833202    4521 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833204    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833207    4521 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833210    4521 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833212    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833215    4521 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833218    4521 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:13.836219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833220    4521 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833223    4521 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833225    4521 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833228    4521 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833230    4521 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833233    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833236    4521 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833238    4521 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833241    4521 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833244    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833246    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833248    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833251    4521 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833254    4521 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833256    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833260    4521 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833264    4521 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833267    4521 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833269    4521 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833272    4521 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:13.836698 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833275    4521 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833277    4521 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833280    4521 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833282    4521 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833285    4521 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833287    4521 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833297    4521 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833300    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833302    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833305    4521 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833307    4521 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833310    4521 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833312    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833315    4521 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833856    4521 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833863    4521 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833866    4521 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833869    4521 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833872    4521 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833875    4521 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:13.837222 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833877    4521 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833880    4521 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833883    4521 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833886    4521 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833890    4521 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833893    4521 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833896    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833898    4521 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833901    4521 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833904    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833907    4521 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833909    4521 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833912    4521 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833915    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833917    4521 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833920    4521 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833923    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833926    4521 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833934    4521 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:13.837715 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833937    4521 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833940    4521 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833942    4521 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833945    4521 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833947    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833950    4521 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833952    4521 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833955    4521 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833959    4521 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833963    4521 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833967    4521 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833970    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833973    4521 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833976    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833979    4521 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833981    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833984    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833986    4521 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833989    4521 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833991    4521 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:13.838236 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833994    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.833997    4521 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834000    4521 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834002    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834005    4521 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834007    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834010    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834013    4521 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834016    4521 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834018    4521 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834020    4521 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834023    4521 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834031    4521 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834034    4521 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834036    4521 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834039    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834041    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834044    4521 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834046    4521 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:13.838726 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834049    4521 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834051    4521 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834054    4521 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834057    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834059    4521 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834064    4521 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834067    4521 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834070    4521 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834073    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834076    4521 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834079    4521 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834081    4521 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834084    4521 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834087    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834089    4521 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834092    4521 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834095    4521 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834097    4521 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834100    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:13.839204 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834102    4521 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834105    4521 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834107    4521 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834110    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834113    4521 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834115    4521 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834118    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834121    4521 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834124    4521 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834126    4521 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834128    4521 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834131    4521 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834134    4521 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834136    4521 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834139    4521 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834142    4521 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834144    4521 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834147    4521 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834149    4521 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834152    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:13.839825 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834155    4521 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834158    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834161    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834257    4521 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834264    4521 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834270    4521 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834274    4521 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834282    4521 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834285    4521 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834289    4521 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834293    4521 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834296    4521 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834299    4521 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834303    4521 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834306    4521 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834310    4521 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834313    4521 flags.go:64] FLAG: --cgroup-root=""
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834316    4521 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834319    4521 flags.go:64] FLAG: --client-ca-file=""
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834322    4521 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834325    4521 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834329    4521 flags.go:64] FLAG: --cluster-domain=""
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834332    4521 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 03:17:13.840650 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834336    4521 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834339    4521 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834342    4521 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834346    4521 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834349    4521 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834353    4521 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834356    4521 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834359    4521 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834362    4521 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834365    4521 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834368    4521 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834371    4521 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834376    4521 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834379    4521 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834382    4521 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834385    4521 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834390    4521 flags.go:64] FLAG: --enable-server="true"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834393    4521 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834398    4521 flags.go:64] FLAG: --event-burst="100"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834402    4521 flags.go:64] FLAG: --event-qps="50"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834405    4521 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834408    4521 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834411    4521 flags.go:64] FLAG: --eviction-hard=""
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834415    4521 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 03:17:13.841616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834418    4521 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834422    4521 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834425    4521 flags.go:64] FLAG: --eviction-soft=""
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834429    4521 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834432    4521 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834435    4521 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834438    4521 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834441    4521 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834444    4521 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834447    4521 flags.go:64] FLAG: --feature-gates=""
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834459    4521 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834463    4521 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834466    4521 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834470    4521 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834473    4521 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834476    4521 flags.go:64] FLAG: --help="false"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834479    4521 flags.go:64] FLAG: --hostname-override="ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834482    4521 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834485    4521 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834488    4521 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834492    4521 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834495    4521 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834498    4521 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 03:17:13.842788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834501    4521 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834505    4521 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834508    4521 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834511    4521 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834515    4521 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834518    4521 flags.go:64] FLAG: --kube-reserved=""
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834521    4521 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834523    4521 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834526    4521 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834530    4521 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834533    4521 flags.go:64] FLAG: --lock-file=""
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834536    4521 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834539    4521 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834542    4521 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834548    4521 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834551    4521 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834555    4521 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834558    4521 flags.go:64] FLAG: --logging-format="text"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834561    4521 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834564    4521 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834567    4521 flags.go:64] FLAG: --manifest-url=""
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834570    4521 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834575    4521 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834578    4521 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834582    4521 flags.go:64] FLAG: --max-pods="110"
Nov 04 03:17:13.843912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834585    4521 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834588    4521 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834591    4521 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834594    4521 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834597    4521 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834600    4521 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834603    4521 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834610    4521 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834615    4521 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834619    4521 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834622    4521 flags.go:64] FLAG: --pod-cidr=""
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834625    4521 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834628    4521 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834633    4521 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834636    4521 flags.go:64] FLAG: --port="10250"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834639    4521 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834642    4521 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-04a705e1529bc9032"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834645    4521 flags.go:64] FLAG: --qos-reserved=""
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834648    4521 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834651    4521 flags.go:64] FLAG: --register-node="true"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834654    4521 flags.go:64] FLAG: --register-with-taints=""
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834658    4521 flags.go:64] FLAG: --registry-burst="10"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834661    4521 flags.go:64] FLAG: --registry-qps="5"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834665    4521 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834668    4521 flags.go:64] FLAG: --reserved-memory=""
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834672    4521 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 03:17:13.845005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834675    4521 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834678    4521 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834681    4521 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834684    4521 flags.go:64] FLAG: --runonce="false"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834687    4521 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834690    4521 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834693    4521 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834696    4521 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834699    4521 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834702    4521 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834705    4521 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834708    4521 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834711    4521 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834714    4521 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834719    4521 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834722    4521 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834725    4521 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834728    4521 flags.go:64] FLAG: --system-cgroups=""
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834731    4521 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834738    4521 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834741    4521 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834744    4521 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834747    4521 flags.go:64] FLAG: --tls-min-version=""
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834750    4521 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834753    4521 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 03:17:13.846152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834756    4521 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834759    4521 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834762    4521 flags.go:64] FLAG: --v="2"
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834767    4521 flags.go:64] FLAG: --version="false"
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834771    4521 flags.go:64] FLAG: --vmodule=""
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834775    4521 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.834779    4521 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834941    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834946    4521 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834949    4521 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834952    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834955    4521 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834958    4521 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834960    4521 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834963    4521 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834965    4521 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834968    4521 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834971    4521 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834974    4521 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834976    4521 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834979    4521 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:13.847275 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834983    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834986    4521 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834989    4521 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834991    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834994    4521 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.834998    4521 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835001    4521 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835004    4521 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835006    4521 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835009    4521 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835011    4521 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835014    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835017    4521 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835019    4521 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835022    4521 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835024    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835027    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835030    4521 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835033    4521 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:13.848158 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835036    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835038    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835041    4521 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835044    4521 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835046    4521 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835049    4521 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835052    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835054    4521 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835057    4521 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835059    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835062    4521 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835065    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835067    4521 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835072    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835074    4521 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835077    4521 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835081    4521 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835085    4521 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835089    4521 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:13.848956 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835092    4521 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835095    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835098    4521 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835100    4521 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835103    4521 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835105    4521 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835108    4521 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835111    4521 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835113    4521 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835116    4521 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835118    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835121    4521 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835124    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835135    4521 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835138    4521 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835140    4521 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835144    4521 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835148    4521 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835151    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835154    4521 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:13.849614 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835156    4521 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835159    4521 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835162    4521 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835164    4521 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835167    4521 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835170    4521 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835173    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835176    4521 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835178    4521 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835181    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835184    4521 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835187    4521 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835190    4521 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835193    4521 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835195    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835198    4521 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835200    4521 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835203    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835206    4521 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835208    4521 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:13.850311 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835211    4521 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:13.850806 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835213    4521 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:13.850806 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835216    4521 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:13.850806 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835218    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:13.850806 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835221    4521 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:13.850806 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835223    4521 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:13.850806 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835231    4521 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:13.850806 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835234    4521 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:13.850806 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835237    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:13.850806 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835240    4521 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:13.850806 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835242    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:13.850806 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835245    4521 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:13.850806 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835248    4521 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:13.850806 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835250    4521 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:13.850806 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.835302    4521 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:17:13.850806 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835361    4521 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:13.850806 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835365    4521 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835368    4521 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835371    4521 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835374    4521 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835376    4521 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835379    4521 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835382    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835385    4521 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835389    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835391    4521 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835394    4521 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835397    4521 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835401    4521 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835405    4521 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835408    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835411    4521 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835414    4521 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835417    4521 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835419    4521 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:13.851219 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835422    4521 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835424    4521 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835427    4521 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835430    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835438    4521 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835441    4521 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835444    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835446    4521 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835449    4521 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835451    4521 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835454    4521 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835456    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835459    4521 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835461    4521 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835464    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835467    4521 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835469    4521 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835472    4521 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835474    4521 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835477    4521 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:13.851790 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835480    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835484    4521 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835487    4521 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835489    4521 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835492    4521 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835494    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835497    4521 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835499    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835502    4521 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835505    4521 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835507    4521 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835510    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835513    4521 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835515    4521 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835518    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835520    4521 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835523    4521 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835545    4521 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835548    4521 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835551    4521 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:13.852479 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835553    4521 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835562    4521 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835565    4521 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835567    4521 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835570    4521 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835573    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835575    4521 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835578    4521 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835581    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835583    4521 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835586    4521 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835588    4521 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835605    4521 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835611    4521 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835615    4521 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835619    4521 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835624    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835628    4521 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835632    4521 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:13.853048 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835636    4521 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835640    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835644    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835648    4521 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835653    4521 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835657    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835662    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835666    4521 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835670    4521 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835674    4521 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835679    4521 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835700    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835703    4521 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835706    4521 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835709    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835712    4521 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835714    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835717    4521 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835720    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835722    4521 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:13.853513 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835727    4521 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835732    4521 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835736    4521 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835739    4521 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835742    4521 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835746    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.835751    4521 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.840685    4521 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.841401    4521 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.841428    4521 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.841437    4521 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841540    4521 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841549    4521 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841556    4521 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841561    4521 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841566    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841570    4521 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841574    4521 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841580    4521 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:13.854118 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841584    4521 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841590    4521 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841594    4521 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841599    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841603    4521 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841607    4521 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841612    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841616    4521 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841620    4521 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841624    4521 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841630    4521 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841636    4521 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841640    4521 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841645    4521 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841650    4521 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841654    4521 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841661    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841666    4521 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841670    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:13.854602 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841674    4521 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841681    4521 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841688    4521 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841695    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841699    4521 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841704    4521 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841708    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841712    4521 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841717    4521 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841722    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841726    4521 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841731    4521 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841736    4521 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841740    4521 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841745    4521 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841749    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841753    4521 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841757    4521 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841762    4521 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:13.855080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841766    4521 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841771    4521 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841775    4521 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841779    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841785    4521 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841789    4521 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841794    4521 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841798    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841802    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841806    4521 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841811    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841816    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841820    4521 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841825    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841829    4521 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841834    4521 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841838    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841863    4521 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841869    4521 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841873    4521 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:13.855546 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841877    4521 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841881    4521 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841886    4521 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841891    4521 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841895    4521 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841899    4521 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841904    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841908    4521 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841912    4521 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841916    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841921    4521 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841925    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841930    4521 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841944    4521 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841949    4521 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841953    4521 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841959    4521 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841963    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841967    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841971    4521 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:13.856054 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841976    4521 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841980    4521 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841985    4521 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841990    4521 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841994    4521 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.841998    4521 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842003    4521 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842007    4521 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842012    4521 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842016    4521 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842021    4521 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842025    4521 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842029    4521 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842034    4521 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842038    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842043    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842047    4521 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842051    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842056    4521 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:13.856543 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842060    4521 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:13.857047 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.842108    4521 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:17:13.857047 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842334    4521 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:13.857047 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842344    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:13.857047 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842349    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:13.857047 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842353    4521 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:13.857047 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842357    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:13.857047 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842370    4521 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:13.857047 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842375    4521 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:13.857047 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842380    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:13.857047 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842384    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:13.857047 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842388    4521 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:13.857047 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842393    4521 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:13.857047 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842398    4521 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:13.857047 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842402    4521 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:13.857047 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842407    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842411    4521 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842416    4521 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842421    4521 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842425    4521 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842430    4521 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842434    4521 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842438    4521 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842443    4521 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842447    4521 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842459    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842464    4521 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842469    4521 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842473    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842478    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842482    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842486    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842490    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842495    4521 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842499    4521 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:13.857423 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842504    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842508    4521 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842512    4521 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842516    4521 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842520    4521 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842526    4521 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842531    4521 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842535    4521 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842540    4521 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842544    4521 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842549    4521 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842555    4521 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842561    4521 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842566    4521 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842571    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842576    4521 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842580    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842584    4521 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842589    4521 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842593    4521 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:13.857915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842597    4521 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842602    4521 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842606    4521 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842610    4521 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842615    4521 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842619    4521 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842624    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842628    4521 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842632    4521 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842636    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842641    4521 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842645    4521 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842649    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842654    4521 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842658    4521 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842662    4521 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842667    4521 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842671    4521 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842677    4521 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842681    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:13.858434 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842686    4521 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842690    4521 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842694    4521 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842699    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842706    4521 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842712    4521 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842717    4521 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842722    4521 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842727    4521 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842731    4521 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842736    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842740    4521 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842744    4521 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842749    4521 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842753    4521 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842757    4521 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842762    4521 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842766    4521 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842770    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:13.858955 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842774    4521 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:13.859412 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842779    4521 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:13.859412 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842783    4521 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:13.859412 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842787    4521 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:13.859412 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842791    4521 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:13.859412 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842794    4521 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:13.859412 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842798    4521 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:13.859412 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842805    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:13.859412 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842809    4521 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:13.859412 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842813    4521 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:13.859412 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842818    4521 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:13.859412 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842822    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:13.859412 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842828    4521 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:13.859412 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842833    4521 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:13.859412 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.842890    4521 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:17:13.859412 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842969    4521 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842978    4521 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842986    4521 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842991    4521 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.842997    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843002    4521 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843007    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843011    4521 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843016    4521 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843020    4521 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843025    4521 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843029    4521 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843033    4521 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843038    4521 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843042    4521 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843046    4521 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843051    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843055    4521 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843060    4521 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:13.859804 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843065    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843069    4521 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843073    4521 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843077    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843082    4521 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843086    4521 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843094    4521 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843098    4521 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843102    4521 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843106    4521 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843111    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843116    4521 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843121    4521 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843125    4521 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843130    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843137    4521 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843144    4521 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843148    4521 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843154    4521 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:13.860266 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843159    4521 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843164    4521 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843168    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843172    4521 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843177    4521 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843181    4521 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843185    4521 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843190    4521 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843195    4521 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843199    4521 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843204    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843208    4521 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843212    4521 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843217    4521 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843221    4521 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843225    4521 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843229    4521 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843234    4521 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843238    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:13.860737 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843244    4521 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843249    4521 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843253    4521 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843258    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843262    4521 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843267    4521 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843274    4521 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843278    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843283    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843287    4521 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843292    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843296    4521 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843301    4521 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843305    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843309    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843313    4521 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843318    4521 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843322    4521 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843327    4521 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:13.861467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843336    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843341    4521 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843345    4521 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843350    4521 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843354    4521 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843359    4521 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843363    4521 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843367    4521 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843371    4521 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843376    4521 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843380    4521 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843385    4521 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843390    4521 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843396    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843400    4521 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843404    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843409    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843413    4521 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843417    4521 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843421    4521 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:13.862168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843427    4521 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:13.862665 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843432    4521 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:13.862665 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843437    4521 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:13.862665 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843441    4521 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:13.862665 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843445    4521 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:13.862665 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843450    4521 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:13.862665 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843456    4521 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:13.862665 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843460    4521 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:13.862665 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843464    4521 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:13.862665 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:13.843469    4521 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:13.862665 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.843549    4521 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 03:17:13.862665 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.845103    4521 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 04 03:17:13.862665 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.845178    4521 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 04 03:17:13.862665 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.845447    4521 server.go:1014] "Starting client certificate rotation"
Nov 04 03:17:13.862665 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.845575    4521 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 03:17:13.862665 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.845622    4521 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 03:11:33 +0000 UTC" deadline="2028-08-11 21:55:46.758488476 +0000 UTC"
Nov 04 03:17:13.862665 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.845650    4521 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15522h38m32.912840631s"
Nov 04 03:17:13.863081 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.846190    4521 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 03:17:13.863081 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.847873    4521 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 03:17:13.863081 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.849603    4521 log.go:25] "Validated CRI v1 runtime API"
Nov 04 03:17:13.863081 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.851551    4521 log.go:25] "Validated CRI v1 image API"
Nov 04 03:17:13.863081 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.852330    4521 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 03:17:13.863081 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.853858    4521 fs.go:135] Filesystem UUIDs: map[4c82457a-93b6-4363-9b2f-48445864594d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a836b55b-aa26-4efc-8c03-ee09c90a57e3:/dev/nvme0n1p4]
Nov 04 03:17:13.863255 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.853880    4521 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0e16d135de0117c644a6f9c809237a412d553e7ed6b63de6fc9e9d168c32f20a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0e16d135de0117c644a6f9c809237a412d553e7ed6b63de6fc9e9d168c32f20a/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/61aede999be255b0ba384dc4cd57becbdfacbd165dfedb10e772c35f813819fd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/61aede999be255b0ba384dc4cd57becbdfacbd165dfedb10e772c35f813819fd/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6d864815b0dbcde352c4c05c2fd96425f4831e85b485d428c78be8e447d07fbc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6d864815b0dbcde352c4c05c2fd96425f4831e85b485d428c78be8e447d07fbc/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6e0704982824dc3fb0713a5fc64778a70fcd7b9ad8bad1b9aaf2dfcd530d119d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6e0704982824dc3fb0713a5fc64778a70fcd7b9ad8bad1b9aaf2dfcd530d119d/userdata/shm major:0 minor:208 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7cbf476f2b9749722e3dcf3981c9f9728e3bcd31d5d1bbb7747647cf8bd15fcc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7cbf476f2b9749722e3dcf3981c9f9728e3bcd31d5d1bbb7747647cf8bd15fcc/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8ff1b16ada0753c352d1a960e89f472017e18d1987395cfc7a61a749576572f2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8ff1b16ada0753c352d1a960e89f472017e18d1987395cfc7a61a749576572f2/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/984183d79b7c2b4c1084d38a28e1d1ffc701595e0a0026b51aacf9c4c62b0ff1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/984183d79b7c2b4c1084d38a28e1d1ffc701595e0a0026b51aacf9c4c62b0ff1/userdata/shm major:0 minor:214 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9c43cf7b698ca862824757a46893033b7767cc57d6863ac457874e47434de10b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9c43cf7b698ca862824757a46893033b7767cc57d6863ac457874e47434de10b/userdata/shm major:0 minor:217 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a6288b478b499a7b2d1ab2950b3bd519717726b9c9276dd9d8ce055b7105d94d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a6288b478b499a7b2d1ab2950b3bd519717726b9c9276dd9d8ce055b7105d94d/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ae0dca1e7493e5e505cebb8085274e51457576c0988a47c3ae240f9702f9879f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ae0dca1e7493e5e505cebb8085274e51457576c0988a47c3ae240f9702f9879f/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bfc3a0432cc1469411ac6bd53ec3a564216e325b84a56bfb8341561fdba04e53/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bfc3a0432cc1469411ac6bd53ec3a564216e325b84a56bfb8341561fdba04e53/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cec18679e27eb17487e79dfdf14c395c949dcdc91fd2f74f8e6d84f0942334a2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cec18679e27eb17487e79dfdf14c395c949dcdc91fd2f74f8e6d84f0942334a2/userdata/shm major:0 minor:219 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e27f857d8e36a049a3a9bdd3de851a12ca9a6447a9ea73fcedff7ce387d215d6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e27f857d8e36a049a3a9bdd3de851a12ca9a6447a9ea73fcedff7ce387d215d6/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ee15f239bf435ca90f74e18f3ab023f838325097e419ce753d62a5c1216ce8b2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ee15f239bf435ca90f74e18f3ab023f838325097e419ce753d62a5c1216ce8b2/userdata/shm major:0 minor:235 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f33e2afd49a996b53b9e3356f66dea2f35333eb4387d664a2f2d30467fb8b42e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f33e2afd49a996b53b9e3356f66dea2f35333eb4387d664a2f2d30467fb8b42e/userdata/shm major:0 minor:64 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ff7a8776f992f8a0d459f0df3ca7b2661e866cf80564a52c87c90d9cef4b5289/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ff7a8776f992f8a0d459f0df3ca7b2661e866cf80564a52c87c90d9cef4b5289/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0924a641-d0e7-4cdb-8130-f17a47a4c026/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/0924a641-d0e7-4cdb-8130-f17a47a4c026/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/16976140-0bb0-4983-bcb7-b2fa2274d24e/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/16976140-0bb0-4983-bcb7-b2fa2274d24e/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/16976140-0bb0-4983-bcb7-b2fa2274d24e/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/16976140-0bb0-4983-bcb7-b2fa2274d24e/volumes/kubernetes.io~empty-dir/tmp major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/16976140-0bb0-4983-bcb7-b2fa2274d24e/volumes/kubernetes.io~projected/kube-api-access-7nnm8:{mountpoint:/var/lib/kubelet/pods/16976140-0bb0-4983-bcb7-b2fa2274d24e/volumes/kubernetes.io~projected/kube-api-access-7nnm8 major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/18329af9-cec7-48fd-aeee-18367bb6c078/volumes/kubernetes.io~projected/kube-api-access-m65mn:{mountpoint:/var/lib/kubelet/pods/18329af9-cec7-48fd-aeee-18367bb6c078/volumes/kubernetes.io~projected/kube-api-access-m65mn major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/23b77dc0-f7c4-4e72-a022-d75d8c8221fd/volumes/kubernetes.io~projected/kube-api-access-7kmrh:{mountpoint:/var/lib/kubelet/pods/23b77dc0-f7c4-4e72-a022-d75d8c8221fd/volumes/kubernetes.io~projected/kube-api-access-7kmrh major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/41f71514-1213-429b-8103-5fb88c9c7039/volumes/kubernetes.io~projected/kube-api-access-7jwdt:{mountpoint:/var/lib/kubelet/pods/41f71514-1213-429b-8103-5fb88c9c7039/volumes/kubernetes.io~projected/kube-api-access-7jwdt major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4db0986b-dec5-4a04-b0f0-ebbc3298f250/volumes/kubernetes.io~projected/kube-api-access-96vcz:{mountpoint:/var/lib/kubelet/pods/4db0986b-dec5-4a04-b0f0-ebbc3298f250/volumes/kubernetes.io~projected/kube-api-access-96vcz major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5d04c72f-9d65-429a-ad6c-dc69054ba75e/volumes/kubernetes.io~projected/kube-api-access-pq5wt:{mountpoint:/var/lib/kubelet/pods/5d04c72f-9d65-429a-ad6c-dc69054ba75e/volumes/kubernetes.io~projected/kube-api-access-pq5wt major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8/volumes/kubernetes.io~projected/kube-api-access-98vqg:{mountpoint:/var/lib/kubelet/pods/62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8/volumes/kubernetes.io~projected/kube-api-access-98vqg major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/70ed447c-add1-4e78-94fb-6aa3e4389fb5/volumes/kubernetes.io~projected/kube-api-access-qmghw:{mountpoint:/var/lib/kubelet/pods/70ed447c-add1-4e78-94fb-6aa3e4389fb5/volumes/kubernetes.io~projected/kube-api-access-qmghw major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7721e70f-2dda-400b-b1a7-d434249efb7e/volumes/kubernetes.io~projected/kube-api-access-wdvfm:{mountpoint:/var/lib/kubelet/pods/7721e70f-2dda-400b-b1a7-d434249efb7e/volumes/kubernetes.io~projected/kube-api-access-wdvfm major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7e6f6098-4287-4076-a8c5-2e55067dfb87/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/7e6f6098-4287-4076-a8c5-2e55067dfb87/volumes/kubernetes.io~secret/agent-certs major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/97184dbe-af1d-408e-8dbc-0822eb2f2d8d/volumes/kubernetes.io~projected/kube-api-access-bmgmq:{mountpoint:/var/lib/kubelet/pods/97184dbe-af1d-408e-8dbc-0822eb2f2d8d/volumes/kubernetes.io~projected/kube-api-access-bmgmq major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a47538cd-f8b8-409f-8eed-c0d9881a822e/volumes/kubernetes.io~projected/kube-api-access-86ljh:{mountpoint:/var/lib/kubelet/pods/a47538cd-f8b8-409f-8eed-c0d9881a822e/volumes/kubernetes.io~projected/kube-api-access-86ljh major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a4fa5e68-1a28-41d2-9680-c79f0d95e361/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/a4fa5e68-1a28-41d2-9680-c79f0d95e361/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a4fa5e68-1a28-41d2-9680-c79f0d95e361/volumes/kubernetes.io~projected/kube-api-access-vmbrz:{mountpoint:/var/lib/kubelet/pods/a4fa5e68-1a28-41d2-9680-c79f0d95e361/volumes/kubernetes.io~projected/kube-api-access-vmbrz major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a4fa5e68-1a28-41d2-9680-c79f0d95e361/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/a4fa5e68-1a28-41d2-9680-c79f0d95e361/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab11857e-7c79-464b-aba7-c6df7ef55a7f/volumes/kubernetes.io~projected/kube-api-access-9xts4:{mountpoint:/var/lib/kubelet/pods/ab11857e-7c79-464b-aba7-c6df7ef55a7f/volumes/kubernetes.io~projected/kube-api-access-9xts4 major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab11857e-7c79-464b-aba7-c6df7ef55a7f/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/ab11857e-7c79-464b-aba7-c6df7ef55a7f/volumes/kubernetes.io~secret/signing-key major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ae6f9681-26d2-4340-8b43-fd6092388459/volumes/kubernetes.io~projected/kube-api-access-jvrhn:{mountpoint:/var/lib/kubelet/pods/ae6f9681-26d2-4340-8b43-fd6092388459/volumes/kubernetes.io~projected/kube-api-access-jvrhn major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ae6f9681-26d2-4340-8b43-fd6092388459/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/ae6f9681-26d2-4340-8b43-fd6092388459/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b7f9ded5-2460-4131-ab22-85c462aaae48/volumes/kubernetes.io~projected/kube-api-access-hgp4k:{mountpoint:/var/lib/kubelet/pods/b7f9ded5-2460-4131-ab22-85c462aaae48/volumes/kubernetes.io~projected/kube-api-access-hgp4k major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ea8cb5af-3757-47db-840e-728d8d0ad62d/volumes/kubernetes.io~projected/kube-api-access-dmw2d:{mountpoint:/var/lib/kubelet/pods/ea8cb5af-3757-47db-840e-728d8d0ad62d/volumes/kubernetes.io~projected/kube-api-access-dmw2d major:0 minor:55 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/272f0c9fde0ed3430f62a80b5c6a7a1d76d3b127a5594337dd26b6dbd8bf83be/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/a4b54c2fdf0bac6833391e29e69a4f718a5b7b7d79283e3d42b31a26cec92324/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-105:{mountpoint:/var/lib/containers/storage/overlay/20bf403f36ae26335eea68fdbd9b3ea2e39c540e059901e5b8234401480e457b/merged major:0 minor:105 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/82e13a75ae1d95b1e9b0af10fc91e9f9dc761c8bafff5b6941ef44740be45c27/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/53154dddcd61dd9c25298e3fd644c75f9966a10c63df3d900edb72f8b969fdac/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/49e2f67a3769ec9746f046f34bd01f693a5567427a24ca093e27228db7ca9534/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/8ca298061dcbd50cf00132fe6b48a2b9dffcd91a603e4b2d3f7a16a1234c9bb3/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/3b5076a982699ea5d693ce0aeac0cb4635a94e1b39bc00c81dc3bd0ba790cb4e/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-131:{mountpoint:/var/lib/containers/storage/overlay/b86cac3f514311e52dc71e4f21f86f77592023b4bab9589fb2086706547ddfc4/merged major:0 minor:131 fsType:overlay blockSize:0} overlay_0-133:{mountpoint:/var/lib/containers/storage/overlay/ac668b3c7c348f1cb3824e30cd99139664a3b88cca75111d4b34347fa6910459/merged major:0 minor:133 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/0402661cc431a2df0cd2e72b114c617bf8bce4a1f0111011225a148334efb9d8/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/f9dc0ac15122bcae0f27086a34480c8eb28e878a56646e33f9464010dc90ea92/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/c19f275a70b55c9a3aa571c49b79a49511f9eed39226903c0394f4a5454f14e9/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/58d280a6f9d88baf53e3963a2c1dcd2d5602ab7fa05668b39b6f179d987cd986/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/b0a286b2c93e88da780e8e6e0b7c004154e5b546ab27bec73057916a65d3724a/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/5f3c9b71d53faf3c6a87b9d581f77959b539987f95ab90c2678ed5614c736daa/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/a266bb267860c64a08361900bb34d8f1954b6cb733cde274682b68f9a7f3bcbb/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/dda5d9d709ecad932bd2d2fcefb1d31fbdf6cb8ac08466134da74988e8831246/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/f35dd183ed504d78bc39533a06145cc54dc9f8003183a05638a77de81cf382d9/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/c4e2ab489612de405013a2ddf63403ba07ab58ea75bd486c95a15555836d481b/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/d4382fb51e1a90ab536c1815fa7bca9212b6a722a9594bf7404db535bed3d740/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-204:{mountpoint:/var/lib/containers/storage/overlay/373bf7560eaf5b6f0528af4074c46bb971d57e84001838d1278116fa13ab609b/merged major:0 minor:204 fsType:overlay blockSize:0} overlay_0-210:{mountpoint:/var/lib/containers/storage/overlay/622f9adb40c0729b1b91f3d3ffe8f27198739317a7523d65ea7fd496637cd0bf/merged major:0 minor:210 fsType:overlay blockSize:0} overlay_0-221:{mountpoint:/var/lib/containers/storage/overlay/f79cd377a246038b4655600f09c477ce5023fec2e6cb642463e46a5fc5b32be3/merged major:0 minor:221 fsType:overlay blockSize:0} overlay_0-223:{mountpoint:/var/lib/containers/storage/overlay/84c08b2d52c2837003e198f951138c84f53c05f0e62bec0a551fada70d5cb545/merged major:0 minor:223 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/b052b8e63383f6e05e031b4e4914f5366474577d13fe76f344f8d58c4bb3ff5e/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-227:{mountpoint:/var/lib/containers/storage/overlay/b653ce6df4b929474d48ad5090ca46bffd278416561a43836b1bb03f59cafa99/merged major:0 minor:227 fsType:overlay blockSize:0} overlay_0-237:{mountpoint:/var/lib/containers/storage/overlay/e7cbf38758da9931f0801da6396a8b093911a59ac13ca6d1123ab16b48aa0f7d/merged major:0 minor:237 fsType:overlay blockSize:0} overlay_0-239:{mountpoint:/var/lib/containers/storage/overlay/8d88d8fdc5766704e27a7013139825069c280843f8ad29374635d50e43900152/merged major:0 minor:239 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/f67853fc5dfb57ce641376bcea515847ac3848c6a39b8104eb6784dfa29a7de0/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/87beccde3a34d20e5668b5a93b816577922efaeb001fbb032335d856c15348b6/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/2bf45b738f4d3af1ddfa430b53b788ba0ab4c11d1ec706eb52e115e31a4f0ac7/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/e38d9813528486f39442a6022b7f46ddbf41773614762466228f5fbd3e0f4f80/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-62:{mountpoint:/var/lib/containers/storage/overlay/23b2403cc89e84201d1157f97f383051308b6264822fb56eab670a23cb3088d3/merged major:0 minor:62 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/7a2dd11d4ca34af7c3839c94859439e819c513af4bf19a9ed8acc761b8680e3d/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/5a098d0a334c662d96bc6b694a757b42e456707ecc6e441694b5d238b73fd13b/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/b9548b18f4c9cd14bcd1c13277e1d5216dfbd799ad1c378fcaefd278a60d0260/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-79:{mountpoint:/var/lib/containers/storage/overlay/b42b4810ba79a81b8ceb7e2feb7f46b5026f6d3ff1a591684985ac484b214869/merged major:0 minor:79 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/035f598c203f46aa3b1b835fb343fa760862ce3be8c00a7d01945f20b1e9278d/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/b487121e19ab1a51d0262b1f55bc5fcf8744d04b4d357a69020fed26073b0d69/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-89:{mountpoint:/var/lib/containers/storage/overlay/046c1ffe9f23bebc15782ae1bf52e1b58f84f5a6fee7001a5bb3713a16d169d5/merged major:0 minor:89 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/4d9d086ad3219a46fbf69a1db4869c82d10c6da9228b13afe872c0a2739da77b/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/eb7216188611cfcd64f31d68e4ca3820d416a0e9622eacaf4158f1c4a81e9a1c/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 04 03:17:13.863407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.862106    4521 manager.go:217] Machine: {Timestamp:2026-11-04 03:17:13.860188537 +0000 UTC m=+0.064788832 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3134484 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ca891bee4ef787ecbe98ad4d899d3 SystemUUID:ec2ca891-bee4-ef78-7ecb-e98ad4d899d3 BootID:950e00b3-79cb-475d-9ae5-cf1007dbeb72 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/7e6f6098-4287-4076-a8c5-2e55067dfb87/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:50 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-210 DeviceMajor:0 DeviceMinor:210 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-223 DeviceMajor:0 DeviceMinor:223 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-227 DeviceMajor:0 DeviceMinor:227 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7cbf476f2b9749722e3dcf3981c9f9728e3bcd31d5d1bbb7747647cf8bd15fcc/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ff7a8776f992f8a0d459f0df3ca7b2661e866cf80564a52c87c90d9cef4b5289/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-133 DeviceMajor:0 DeviceMinor:133 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8/volumes/kubernetes.io~projected/kube-api-access-98vqg DeviceMajor:0 DeviceMinor:213 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bfc3a0432cc1469411ac6bd53ec3a564216e325b84a56bfb8341561fdba04e53/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ab11857e-7c79-464b-aba7-c6df7ef55a7f/volumes/kubernetes.io~projected/kube-api-access-9xts4 DeviceMajor:0 DeviceMinor:234 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-237 DeviceMajor:0 DeviceMinor:237 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/18329af9-cec7-48fd-aeee-18367bb6c078/volumes/kubernetes.io~projected/kube-api-access-m65mn DeviceMajor:0 DeviceMinor:53 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e27f857d8e36a049a3a9bdd3de851a12ca9a6447a9ea73fcedff7ce387d215d6/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8ff1b16ada0753c352d1a960e89f472017e18d1987395cfc7a61a749576572f2/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0924a641-d0e7-4cdb-8130-f17a47a4c026/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:212 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/41f71514-1213-429b-8103-5fb88c9c7039/volumes/kubernetes.io~projected/kube-api-access-7jwdt DeviceMajor:0 DeviceMinor:206 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9c43cf7b698ca862824757a46893033b7767cc57d6863ac457874e47434de10b/userdata/shm DeviceMajor:0 DeviceMinor:217 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cec18679e27eb17487e79dfdf14c395c949dcdc91fd2f74f8e6d84f0942334a2/userdata/shm DeviceMajor:0 DeviceMinor:219 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-239 DeviceMajor:0 DeviceMinor:239 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/16976140-0bb0-4983-bcb7-b2fa2274d24e/volumes/kubernetes.io~projected/kube-api-access-7nnm8 DeviceMajor:0 DeviceMinor:51 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/23b77dc0-f7c4-4e72-a022-d75d8c8221fd/volumes/kubernetes.io~projected/kube-api-access-7kmrh DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-131 DeviceMajor:0 DeviceMinor:131 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/16976140-0bb0-4983-bcb7-b2fa2274d24e/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:48 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a4fa5e68-1a28-41d2-9680-c79f0d95e361/volumes/kubernetes.io~projected/kube-api-access-vmbrz DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-89 DeviceMajor:0 DeviceMinor:89 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/97184dbe-af1d-408e-8dbc-0822eb2f2d8d/volumes/kubernetes.io~projected/kube-api-access-bmgmq DeviceMajor:0 DeviceMinor:207 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/984183d79b7c2b4c1084d38a28e1d1ffc701595e0a0026b51aacf9c4c62b0ff1/userdata/shm DeviceMajor:0 DeviceMinor:214 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-62 DeviceMajor:0 DeviceMinor:62 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/61aede999be255b0ba384dc4cd57becbdfacbd165dfedb10e772c35f813819fd/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ae6f9681-26d2-4340-8b43-fd6092388459/volumes/kubernetes.io~projected/kube-api-access-jvrhn DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b7f9ded5-2460-4131-ab22-85c462aaae48/volumes/kubernetes.io~projected/kube-api-access-hgp4k DeviceMajor:0 DeviceMinor:215 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/70ed447c-add1-4e78-94fb-6aa3e4389fb5/volumes/kubernetes.io~projected/kube-api-access-qmghw DeviceMajor:0 DeviceMinor:54 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ae6f9681-26d2-4340-8b43-fd6092388459/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6e0704982824dc3fb0713a5fc64778a70fcd7b9ad8bad1b9aaf2dfcd530d119d/userdata/shm DeviceMajor:0 DeviceMinor:208 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ee15f239bf435ca90f74e18f3ab023f838325097e419ce753d62a5c1216ce8b2/userdata/shm DeviceMajor:0 DeviceMinor:235 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/16976140-0bb0-4983-bcb7-b2fa2274d24e/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:49 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ae0dca1e7493e5e505cebb8085274e51457576c0988a47c3ae240f9702f9879f/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0e16d135de0117c644a6f9c809237a412d553e7ed6b63de6fc9e9d168c32f20a/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a4fa5e68-1a28-41d2-9680-c79f0d95e361/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-204 DeviceMajor:0 DeviceMinor:204 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ea8cb5af-3757-47db-840e-728d8d0ad62d/volumes/kubernetes.io~projected/kube-api-access-dmw2d DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-79 DeviceMajor:0 DeviceMinor:79 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ab11857e-7c79-464b-aba7-c6df7ef55a7f/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:229 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5d04c72f-9d65-429a-ad6c-dc69054ba75e/volumes/kubernetes.io~projected/kube-api-access-pq5wt DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a47538cd-f8b8-409f-8eed-c0d9881a822e/volumes/kubernetes.io~projected/kube-api-access-86ljh DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-221 DeviceMajor:0 DeviceMinor:221 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a4fa5e68-1a28-41d2-9680-c79f0d95e361/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:47 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a6288b478b499a7b2d1ab2950b3bd519717726b9c9276dd9d8ce055b7105d94d/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4db0986b-dec5-4a04-b0f0-ebbc3298f250/volumes/kubernetes.io~projected/kube-api-access-96vcz DeviceMajor:0 DeviceMinor:52 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7721e70f-2dda-400b-b1a7-d434249efb7e/volumes/kubernetes.io~projected/kube-api-access-wdvfm DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-105 DeviceMajor:0 DeviceMinor:105 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6d864815b0dbcde352c4c05c2fd96425f4831e85b485d428c78be8e447d07fbc/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f33e2afd49a996b53b9e3356f66dea2f35333eb4387d664a2f2d30467fb8b42e/userdata/shm DeviceMajor:0 DeviceMinor:64 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:6d864815b0dbcde MacAddress:ea:62:93:fb:87:74 Speed:10000 Mtu:8901} {Name:6e0704982824dc3 MacAddress:96:af:fc:a7:d1:e4 Speed:10000 Mtu:8901} {Name:984183d79b7c2b4 MacAddress:6a:60:66:16:93:d7 Speed:10000 Mtu:8901} {Name:9c43cf7b698ca86 MacAddress:be:8e:2d:28:5f:e0 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:f2:2d:17:31 Speed:0 Mtu:9001} {Name:br-int MacAddress:e6:d1:b4:07:62:bb Speed:0 Mtu:8901} {Name:cec18679e27eb17 MacAddress:52:15:f1:c2:05:c0 Speed:10000 Mtu:8901} {Name:ee15f239bf435ca MacAddress:da:bd:1f:c4:5c:3b Speed:10000 Mtu:8901} {Name:ens5 MacAddress:06:ff:f2:2d:17:31 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:b2:ab:d3:5e:79:9e Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:b2:97:13:c9:eb:e9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 03:17:13.863407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.862401    4521 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 03:17:13.863407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.862467    4521 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 03:17:13.863407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.862761    4521 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 03:17:13.863407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.862789    4521 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-242.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 03:17:13.863407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.862931    4521 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 03:17:13.863407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.862938    4521 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 03:17:13.863407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.862949    4521 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 03:17:13.863407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.862967    4521 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 03:17:13.863407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.863055    4521 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 03:17:13.863407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.863211    4521 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 03:17:13.863407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.863323    4521 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 03:17:13.863407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.863360    4521 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 03:17:13.863407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.863379    4521 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 03:17:13.863407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.863396    4521 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 03:17:13.863407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.863409    4521 kubelet.go:405] "Adding apiserver pod source"
Nov 04 03:17:13.864223 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.863420    4521 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 03:17:13.864223 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.864151    4521 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 03:17:13.864223 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.864162    4521 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 03:17:13.864665 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.864647    4521 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 03:17:13.866616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.866597    4521 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 03:17:13.866616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.866610    4521 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 03:17:13.866747 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.866711    4521 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 04 03:17:13.866920 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.866905    4521 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 03:17:13.866986 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.866935    4521 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 03:17:13.867031 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.867026    4521 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 03:17:13.867064 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.867039    4521 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 03:17:13.867064 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.867045    4521 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 03:17:13.867064 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.867051    4521 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 03:17:13.867064 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.867056    4521 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 03:17:13.867064 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.867061    4521 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 03:17:13.867243 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.867071    4521 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 03:17:13.867243 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.867078    4521 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 03:17:13.867243 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.867084    4521 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 03:17:13.867243 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.867090    4521 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 03:17:13.867243 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.867103    4521 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 03:17:13.867243 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.867112    4521 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 03:17:13.867243 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.867138    4521 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 03:17:13.867243 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.867144    4521 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 03:17:13.870475 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.870462    4521 server.go:1263] "Started kubelet"
Nov 04 03:17:13.871094 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.870533    4521 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 03:17:13.871094 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.870991    4521 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 03:17:13.871231 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.871079    4521 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 03:17:13.871512 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.871456    4521 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 03:17:13.871585 ip-10-0-135-242 systemd[1]: Started Kubernetes Kubelet.
Nov 04 03:17:13.872227 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.872181    4521 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 03:17:13.875362 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.875336    4521 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 03:17:13.875672 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.875421    4521 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 03:17:13.875672 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.875531    4521 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 03:17:13.875672 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.875547    4521 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 03:17:13.875672 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.875600    4521 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 03:17:13.877350 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877297    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab11857e-7c79-464b-aba7-c6df7ef55a7f" volumeName="kubernetes.io/projected/ab11857e-7c79-464b-aba7-c6df7ef55a7f-kube-api-access-9xts4" seLinuxMountContext=""
Nov 04 03:17:13.877418 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877356    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16976140-0bb0-4983-bcb7-b2fa2274d24e" volumeName="kubernetes.io/empty-dir/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-tuned" seLinuxMountContext=""
Nov 04 03:17:13.877418 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877371    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23b77dc0-f7c4-4e72-a022-d75d8c8221fd" volumeName="kubernetes.io/projected/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-kube-api-access-7kmrh" seLinuxMountContext=""
Nov 04 03:17:13.877418 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877384    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a4fa5e68-1a28-41d2-9680-c79f0d95e361" volumeName="kubernetes.io/projected/a4fa5e68-1a28-41d2-9680-c79f0d95e361-kube-api-access-vmbrz" seLinuxMountContext=""
Nov 04 03:17:13.877418 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877398    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae6f9681-26d2-4340-8b43-fd6092388459" volumeName="kubernetes.io/secret/ae6f9681-26d2-4340-8b43-fd6092388459-cert" seLinuxMountContext=""
Nov 04 03:17:13.877418 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877413    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8" volumeName="kubernetes.io/projected/62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8-kube-api-access-98vqg" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877425    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70ed447c-add1-4e78-94fb-6aa3e4389fb5" volumeName="kubernetes.io/empty-dir/70ed447c-add1-4e78-94fb-6aa3e4389fb5-tmp-dir" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877441    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7721e70f-2dda-400b-b1a7-d434249efb7e" volumeName="kubernetes.io/configmap/7721e70f-2dda-400b-b1a7-d434249efb7e-cni-binary-copy" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877456    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7e6f6098-4287-4076-a8c5-2e55067dfb87" volumeName="kubernetes.io/configmap/7e6f6098-4287-4076-a8c5-2e55067dfb87-konnectivity-ca" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877469    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a47538cd-f8b8-409f-8eed-c0d9881a822e" volumeName="kubernetes.io/configmap/a47538cd-f8b8-409f-8eed-c0d9881a822e-config-volume" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877482    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16976140-0bb0-4983-bcb7-b2fa2274d24e" volumeName="kubernetes.io/empty-dir/16976140-0bb0-4983-bcb7-b2fa2274d24e-tmp" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877494    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a47538cd-f8b8-409f-8eed-c0d9881a822e" volumeName="kubernetes.io/empty-dir/a47538cd-f8b8-409f-8eed-c0d9881a822e-tmp-dir" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877507    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a47538cd-f8b8-409f-8eed-c0d9881a822e" volumeName="kubernetes.io/projected/a47538cd-f8b8-409f-8eed-c0d9881a822e-kube-api-access-86ljh" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877518    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab11857e-7c79-464b-aba7-c6df7ef55a7f" volumeName="kubernetes.io/configmap/ab11857e-7c79-464b-aba7-c6df7ef55a7f-signing-cabundle" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877532    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="18329af9-cec7-48fd-aeee-18367bb6c078" volumeName="kubernetes.io/configmap/18329af9-cec7-48fd-aeee-18367bb6c078-serviceca" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877556    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4db0986b-dec5-4a04-b0f0-ebbc3298f250" volumeName="kubernetes.io/projected/4db0986b-dec5-4a04-b0f0-ebbc3298f250-kube-api-access-96vcz" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877568    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d04c72f-9d65-429a-ad6c-dc69054ba75e" volumeName="kubernetes.io/projected/5d04c72f-9d65-429a-ad6c-dc69054ba75e-kube-api-access-pq5wt" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877580    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70ed447c-add1-4e78-94fb-6aa3e4389fb5" volumeName="kubernetes.io/projected/70ed447c-add1-4e78-94fb-6aa3e4389fb5-kube-api-access-qmghw" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877591    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7721e70f-2dda-400b-b1a7-d434249efb7e" volumeName="kubernetes.io/projected/7721e70f-2dda-400b-b1a7-d434249efb7e-kube-api-access-wdvfm" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877605    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="97184dbe-af1d-408e-8dbc-0822eb2f2d8d" volumeName="kubernetes.io/projected/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-kube-api-access-bmgmq" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877615    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ea8cb5af-3757-47db-840e-728d8d0ad62d" volumeName="kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-cni-binary-copy" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877625    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d04c72f-9d65-429a-ad6c-dc69054ba75e" volumeName="kubernetes.io/configmap/5d04c72f-9d65-429a-ad6c-dc69054ba75e-iptables-alerter-script" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877637    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="97184dbe-af1d-408e-8dbc-0822eb2f2d8d" volumeName="kubernetes.io/empty-dir/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-archives-path" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877651    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a4fa5e68-1a28-41d2-9680-c79f0d95e361" volumeName="kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovnkube-script-lib" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877662    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a4fa5e68-1a28-41d2-9680-c79f0d95e361" volumeName="kubernetes.io/secret/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877674    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0924a641-d0e7-4cdb-8130-f17a47a4c026" volumeName="kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877685    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="18329af9-cec7-48fd-aeee-18367bb6c078" volumeName="kubernetes.io/projected/18329af9-cec7-48fd-aeee-18367bb6c078-kube-api-access-m65mn" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877696    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab11857e-7c79-464b-aba7-c6df7ef55a7f" volumeName="kubernetes.io/secret/ab11857e-7c79-464b-aba7-c6df7ef55a7f-signing-key" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877711    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ea8cb5af-3757-47db-840e-728d8d0ad62d" volumeName="kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877723    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ea8cb5af-3757-47db-840e-728d8d0ad62d" volumeName="kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877735    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ea8cb5af-3757-47db-840e-728d8d0ad62d" volumeName="kubernetes.io/projected/ea8cb5af-3757-47db-840e-728d8d0ad62d-kube-api-access-dmw2d" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877749    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41f71514-1213-429b-8103-5fb88c9c7039" volumeName="kubernetes.io/empty-dir/41f71514-1213-429b-8103-5fb88c9c7039-data-volume" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877752    4521 factory.go:55] Registering systemd factory
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877760    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7e6f6098-4287-4076-a8c5-2e55067dfb87" volumeName="kubernetes.io/secret/7e6f6098-4287-4076-a8c5-2e55067dfb87-agent-certs" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877770    4521 factory.go:223] Registration of the systemd container factory successfully
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877772    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="97184dbe-af1d-408e-8dbc-0822eb2f2d8d" volumeName="kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877785    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a4fa5e68-1a28-41d2-9680-c79f0d95e361" volumeName="kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-env-overrides" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877798    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a4fa5e68-1a28-41d2-9680-c79f0d95e361" volumeName="kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovnkube-config" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877810    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae6f9681-26d2-4340-8b43-fd6092388459" volumeName="kubernetes.io/projected/ae6f9681-26d2-4340-8b43-fd6092388459-kube-api-access-jvrhn" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877823    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b7f9ded5-2460-4131-ab22-85c462aaae48" volumeName="kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877835    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16976140-0bb0-4983-bcb7-b2fa2274d24e" volumeName="kubernetes.io/projected/16976140-0bb0-4983-bcb7-b2fa2274d24e-kube-api-access-7nnm8" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877869    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41f71514-1213-429b-8103-5fb88c9c7039" volumeName="kubernetes.io/projected/41f71514-1213-429b-8103-5fb88c9c7039-kube-api-access-7jwdt" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877881    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7721e70f-2dda-400b-b1a7-d434249efb7e" volumeName="kubernetes.io/configmap/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-daemon-config" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877894    4521 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="97184dbe-af1d-408e-8dbc-0822eb2f2d8d" volumeName="kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-service-ca-bundle" seLinuxMountContext=""
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877906    4521 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.877913    4521 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:13.878169    4521 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-242.us-west-2.compute.internal\" not found"
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.878222    4521 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 03:11:34 +0000 UTC" deadline="2028-04-12 15:46:31.369625538 +0000 UTC"
Nov 04 03:17:13.878534 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.878243    4521 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12612h29m17.491385981s"
Nov 04 03:17:13.879958 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.879945    4521 factory.go:153] Registering CRI-O factory
Nov 04 03:17:13.880019 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.879962    4521 factory.go:223] Registration of the crio container factory successfully
Nov 04 03:17:13.880019 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.880012    4521 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 03:17:13.880101 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.880037    4521 factory.go:103] Registering Raw factory
Nov 04 03:17:13.880101 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.880058    4521 manager.go:1199] Started watching for new ooms in manager
Nov 04 03:17:13.880426 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.880409    4521 manager.go:319] Starting recovery of all containers
Nov 04 03:17:13.902943 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.902914    4521 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 03:17:13.904184 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.904159    4521 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 03:17:13.904184 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.904181    4521 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 03:17:13.904358 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.904197    4521 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 03:17:13.904358 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:13.904235    4521 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 04 03:17:13.912256 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.912220    4521 generic.go:356] "Generic (PLEG): container finished" podID="ea8cb5af-3757-47db-840e-728d8d0ad62d" containerID="3d7c696a092faa900bc031134b69fe1d04f206db5b62511324753d9e5604bdb4" exitCode=0
Nov 04 03:17:13.912256 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.912252    4521 generic.go:356] "Generic (PLEG): container finished" podID="ea8cb5af-3757-47db-840e-728d8d0ad62d" containerID="40d3626a79217c6e2edc4ab867e1c261d83cba0ccbc2ed8d50e584d1df1ee6ea" exitCode=0
Nov 04 03:17:13.912395 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.912260    4521 generic.go:356] "Generic (PLEG): container finished" podID="ea8cb5af-3757-47db-840e-728d8d0ad62d" containerID="812171ba9eafc8aa7f8bf650c8044f746306b01ac107b9634189c72396750aed" exitCode=0
Nov 04 03:17:13.912395 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.912268    4521 generic.go:356] "Generic (PLEG): container finished" podID="ea8cb5af-3757-47db-840e-728d8d0ad62d" containerID="6847ebc17c8471c06fbebfc5a2d1164540b654305a9944baf17c479c155e4d74" exitCode=0
Nov 04 03:17:13.912395 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.912273    4521 generic.go:356] "Generic (PLEG): container finished" podID="ea8cb5af-3757-47db-840e-728d8d0ad62d" containerID="15220f82bf17151512fdfdc57e4cf1cca778350a143c80410bbdf3f3528a46ea" exitCode=0
Nov 04 03:17:13.912395 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.912279    4521 generic.go:356] "Generic (PLEG): container finished" podID="ea8cb5af-3757-47db-840e-728d8d0ad62d" containerID="a96ea1b95b1b9bb184ab07ecbf10578cae2b3af081265251a2b37ea401b00f1e" exitCode=0
Nov 04 03:17:13.926467 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.926433    4521 generic.go:356] "Generic (PLEG): container finished" podID="c879e759b69c22be08682ecf8baecef5" containerID="6c28860e1a7a3a18ef27cb6ff5b1b979164e19954395260149d86f903f773fc2" exitCode=0
Nov 04 03:17:13.950570 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.950547    4521 manager.go:324] Recovery completed
Nov 04 03:17:13.963720 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.963702    4521 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:17:13.964889 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.964872    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:17:13.964962 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.964902    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:17:13.964962 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.964912    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:17:13.966277 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.966262    4521 cpu_manager.go:226] "Starting" policy="none"
Nov 04 03:17:13.966277 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.966274    4521 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 03:17:13.966405 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.966278    4521 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:17:13.966405 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.966290    4521 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 03:17:13.966405 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.966397    4521 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 04 03:17:13.966490 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.966407    4521 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 04 03:17:13.966490 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.966424    4521 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 04 03:17:13.966490 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.966431    4521 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 04 03:17:13.966490 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.966439    4521 policy_none.go:50] "Start"
Nov 04 03:17:13.966490 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.966449    4521 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 03:17:13.966490 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.966467    4521 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 03:17:13.966647 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.966605    4521 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 04 03:17:13.966647 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.966612    4521 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 04 03:17:13.966647 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.966618    4521 policy_none.go:44] "Start"
Nov 04 03:17:13.967486 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.967470    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:17:13.967539 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.967497    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:17:13.967539 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.967518    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:17:13.970907 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.970883    4521 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 03:17:13.973523 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.973509    4521 manager.go:341] "Starting Device Plugin manager"
Nov 04 03:17:13.973607 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:13.973539    4521 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 03:17:13.973607 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.973557    4521 server.go:87] "Starting device plugin registration server"
Nov 04 03:17:13.973837 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.973824    4521 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 03:17:13.973950 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.973927    4521 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 03:17:13.974031 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.974014    4521 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 03:17:13.974193 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.974151    4521 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 03:17:13.974579 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.974551    4521 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 03:17:13.974653 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.974580    4521 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 03:17:13.974770 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.974662    4521 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T03:17:13.974192242Z","UUID":"928dfc33-bba8-4620-9e11-2b2668c57b5a","Handler":null,"Name":"","Endpoint":""}
Nov 04 03:17:13.976561 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:13.976536    4521 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 03:17:13.976644 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:13.976586    4521 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-242.us-west-2.compute.internal\" not found"
Nov 04 03:17:13.976704 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.976676    4521 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 03:17:13.976749 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:13.976716    4521 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 03:17:14.004981 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.004411    4521 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal"]
Nov 04 03:17:14.005066 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.005029    4521 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:17:14.005803 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.005786    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:17:14.005881 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.005813    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:17:14.005881 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.005823    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:17:14.006057 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.006047    4521 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:17:14.006217 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.006202    4521 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:17:14.006766 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.006749    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:17:14.006837 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.006779    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:17:14.006837 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.006751    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:17:14.006837 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.006789    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:17:14.006837 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.006807    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:17:14.006837 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.006819    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:17:14.007017 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:14.006965    4521 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-242.us-west-2.compute.internal\" not found" node="ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.007047 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.007022    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal" event={"ID":"172fe89364b7e525fac46ad33508ae4e","Type":"ContainerStarted","Data":"13b5785c350ae39cb3006159e1ce9b68622ca910b214075c5b036526285477d9"}
Nov 04 03:17:14.007075 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.007051    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal" event={"ID":"172fe89364b7e525fac46ad33508ae4e","Type":"ContainerStarted","Data":"f33e2afd49a996b53b9e3356f66dea2f35333eb4387d664a2f2d30467fb8b42e"}
Nov 04 03:17:14.007075 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.007065    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal" event={"ID":"c879e759b69c22be08682ecf8baecef5","Type":"ContainerStarted","Data":"0c7e801b95c707f52283fc36c1610f5e7baf75826f024b65c68e1e094d98fb4c"}
Nov 04 03:17:14.007126 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.007078    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal" event={"ID":"c879e759b69c22be08682ecf8baecef5","Type":"ContainerDied","Data":"6c28860e1a7a3a18ef27cb6ff5b1b979164e19954395260149d86f903f773fc2"}
Nov 04 03:17:14.007126 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.007091    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal" event={"ID":"c879e759b69c22be08682ecf8baecef5","Type":"ContainerStarted","Data":"e27f857d8e36a049a3a9bdd3de851a12ca9a6447a9ea73fcedff7ce387d215d6"}
Nov 04 03:17:14.007126 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.007102    4521 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:17:14.007748 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.007730    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:17:14.007816 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.007755    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:17:14.007816 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.007769    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:17:14.007911 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:14.007904    4521 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-242.us-west-2.compute.internal\" not found" node="ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.074342 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.074314    4521 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:17:14.075687 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.075663    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:17:14.075768 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.075696    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:17:14.075768 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.075707    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:17:14.075768 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.075725    4521 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.081931 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.081908    4521 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:17:14.082032 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.081911    4521 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:17:14.082032 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.081941    4521 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:17:14.082032 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.082006    4521 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 04 03:17:14.132749 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.132728    4521 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:17:14.176647 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.176623    4521 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.182814 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.182793    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c879e759b69c22be08682ecf8baecef5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"c879e759b69c22be08682ecf8baecef5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.182938 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.182819    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c879e759b69c22be08682ecf8baecef5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"c879e759b69c22be08682ecf8baecef5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.182938 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.182858    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/172fe89364b7e525fac46ad33508ae4e-config\") pod \"kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"172fe89364b7e525fac46ad33508ae4e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.188202 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.188183    4521 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.188285 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.188252    4521 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.188792 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.188777    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeNotReady"
Nov 04 03:17:14.188891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.188805    4521 setters.go:546] "Node became not ready" node="ip-10-0-135-242.us-west-2.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-04T03:17:14Z","lastTransitionTime":"2026-11-04T03:17:14Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Nov 04 03:17:14.254712 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.254677    4521 nodeinfomanager.go:471] existing CSINode "ip-10-0-135-242.us-west-2.compute.internal" is owned by different node (oldNodeID="b3b21e97-beb1-4210-b216-15b590386248", newNodeID=""), cleaning up...
Nov 04 03:17:14.263588 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.263528    4521 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:17:14.263588 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:14.263573    4521 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.263748 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.263591    4521 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.283580 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.283559    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c879e759b69c22be08682ecf8baecef5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"c879e759b69c22be08682ecf8baecef5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.283674 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.283587    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/172fe89364b7e525fac46ad33508ae4e-config\") pod \"kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"172fe89364b7e525fac46ad33508ae4e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.283674 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.283605    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c879e759b69c22be08682ecf8baecef5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"c879e759b69c22be08682ecf8baecef5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.283674 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.283665    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c879e759b69c22be08682ecf8baecef5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"c879e759b69c22be08682ecf8baecef5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.283764 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.283714    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c879e759b69c22be08682ecf8baecef5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"c879e759b69c22be08682ecf8baecef5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.283764 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.283747    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/172fe89364b7e525fac46ad33508ae4e-config\") pod \"kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal\" (UID: \"172fe89364b7e525fac46ad33508ae4e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.324503 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.324481    4521 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:17:14.324567 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:14.324521    4521 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.863929 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.863898    4521 apiserver.go:52] "Watching apiserver"
Nov 04 03:17:14.945909 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.945876    4521 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.946047 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.945941    4521 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:14.969479 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.969447    4521 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 03:17:14.970776 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.970734    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-dm5lv-xwxch","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal","openshift-insights/insights-runtime-extractor-t5zsk","openshift-image-registry/node-ca-tdgzf","openshift-network-diagnostics/network-check-target-6kpt2","kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal","openshift-cluster-node-tuning-operator/tuned-bkcnt","openshift-multus/network-metrics-daemon-lgvq4","openshift-service-ca/service-ca-5fffc97bdf-pnj7z","openshift-dns/node-resolver-qk4nr","openshift-ingress-canary/ingress-canary-xhrpk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4","openshift-kube-storage-version-migrator/migrator-66df5d496b-9fzvp","openshift-ovn-kubernetes/ovnkube-node-zfmg5","openshift-multus/multus-additional-cni-plugins-xrrqq","openshift-multus/multus-hwrqp","openshift-dns/dns-default-vwf9m","openshift-network-operator/iptables-alerter-m9zlr","kube-system/global-pull-secret-syncer-qbcj2","kube-system/konnectivity-agent-sd8pw"]
Nov 04 03:17:14.973013 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.972988    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:17:14.975398 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.975377    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:14.976170 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.976150    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:14.988222 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988201    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:17:14.988325 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988231    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:17:14.988325 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988249    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-run-ovn-kubernetes\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.988325 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988265    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysctl-d\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:14.988325 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988280    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-sys\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:14.988325 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988292    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-host\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:14.988492 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988334    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70ed447c-add1-4e78-94fb-6aa3e4389fb5-tmp-dir\") pod \"node-resolver-qk4nr\" (UID: \"70ed447c-add1-4e78-94fb-6aa3e4389fb5\") " pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:17:14.988492 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988359    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgp4k\" (UniqueName: \"kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k\") pod \"network-check-target-6kpt2\" (UID: \"b7f9ded5-2460-4131-ab22-85c462aaae48\") " pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:17:14.988492 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988398    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70ed447c-add1-4e78-94fb-6aa3e4389fb5-tmp-dir\") pod \"node-resolver-qk4nr\" (UID: \"70ed447c-add1-4e78-94fb-6aa3e4389fb5\") " pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:17:14.988492 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988440    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-cni-bin\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:14.988492 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988456    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-kubelet\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:14.988773 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988548    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-hostroot\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:14.988773 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988599    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-multus-certs\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:14.988773 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988651    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-systemd\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.988773 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988677    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-env-overrides\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.988773 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988707    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvrhn\" (UniqueName: \"kubernetes.io/projected/ae6f9681-26d2-4340-8b43-fd6092388459-kube-api-access-jvrhn\") pod \"ingress-canary-xhrpk\" (UID: \"ae6f9681-26d2-4340-8b43-fd6092388459\") " pod="openshift-ingress-canary/ingress-canary-xhrpk"
Nov 04 03:17:14.988773 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988729    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-cni-netd\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.989006 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988818    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:14.989006 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988834    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-netns\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:14.989006 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988865    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-conf-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:14.989006 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988889    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:14.989006 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988907    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18329af9-cec7-48fd-aeee-18367bb6c078-host\") pod \"node-ca-tdgzf\" (UID: \"18329af9-cec7-48fd-aeee-18367bb6c078\") " pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:17:14.989006 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.988938    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-tuned\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:14.989250 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989026    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-kubelet\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.989250 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989028    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-tuned\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:14.989250 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989061    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.989250 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989092    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-system-cni-dir\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:17:14.989250 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989125    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16976140-0bb0-4983-bcb7-b2fa2274d24e-tmp\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:14.989250 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989152    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:14.989250 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989179    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-socket-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:14.989250 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989190    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16976140-0bb0-4983-bcb7-b2fa2274d24e-tmp\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:14.989250 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989219    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-registration-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:14.989250 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989242    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-device-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:14.989621 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989262    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-run-netns\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.989621 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989280    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86ljh\" (UniqueName: \"kubernetes.io/projected/a47538cd-f8b8-409f-8eed-c0d9881a822e-kube-api-access-86ljh\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:14.989621 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989296    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-k8s-cni-cncf-io\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:14.989621 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989311    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e6f6098-4287-4076-a8c5-2e55067dfb87-agent-certs\") pod \"konnectivity-agent-sd8pw\" (UID: \"7e6f6098-4287-4076-a8c5-2e55067dfb87\") " pod="kube-system/konnectivity-agent-sd8pw"
Nov 04 03:17:14.989621 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989325    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70ed447c-add1-4e78-94fb-6aa3e4389fb5-hosts-file\") pod \"node-resolver-qk4nr\" (UID: \"70ed447c-add1-4e78-94fb-6aa3e4389fb5\") " pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:17:14.989621 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989353    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-ovn\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.989621 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989401    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/41f71514-1213-429b-8103-5fb88c9c7039-data-volume\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:14.989621 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989447    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-modprobe-d\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:14.989621 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989472    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/41f71514-1213-429b-8103-5fb88c9c7039-data-volume\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:14.989621 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989475    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0924a641-d0e7-4cdb-8130-f17a47a4c026-dbus\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:17:14.989621 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989516    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-node-log\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.989621 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989548    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-cni-bin\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.989621 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989572    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nnm8\" (UniqueName: \"kubernetes.io/projected/16976140-0bb0-4983-bcb7-b2fa2274d24e-kube-api-access-7nnm8\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:14.990059 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989670    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:17:14.990059 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989689    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d04c72f-9d65-429a-ad6c-dc69054ba75e-host-slash\") pod \"iptables-alerter-m9zlr\" (UID: \"5d04c72f-9d65-429a-ad6c-dc69054ba75e\") " pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:17:14.990059 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989704    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq5wt\" (UniqueName: \"kubernetes.io/projected/5d04c72f-9d65-429a-ad6c-dc69054ba75e-kube-api-access-pq5wt\") pod \"iptables-alerter-m9zlr\" (UID: \"5d04c72f-9d65-429a-ad6c-dc69054ba75e\") " pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:17:14.990059 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989719    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-sys-fs\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:14.990059 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989734    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kmrh\" (UniqueName: \"kubernetes.io/projected/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-kube-api-access-7kmrh\") pod \"network-metrics-daemon-lgvq4\" (UID: \"23b77dc0-f7c4-4e72-a022-d75d8c8221fd\") " pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:17:14.990059 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989748    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-etc-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.990059 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989949    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovnkube-script-lib\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.990059 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.989978    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/18329af9-cec7-48fd-aeee-18367bb6c078-serviceca\") pod \"node-ca-tdgzf\" (UID: \"18329af9-cec7-48fd-aeee-18367bb6c078\") " pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:17:14.990059 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990003    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-lib-modules\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:14.990059 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990028    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-var-lib-kubelet\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:14.990059 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990056    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-daemon-config\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:14.990514 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990082    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-var-lib-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.990514 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990109    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovn-node-metrics-cert\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.990514 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990212    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-archives-path\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:14.990514 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990236    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:14.990514 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990253    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jwdt\" (UniqueName: \"kubernetes.io/projected/41f71514-1213-429b-8103-5fb88c9c7039-kube-api-access-7jwdt\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:14.990514 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990277    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-archives-path\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:14.990514 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990289    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-socket-dir-parent\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:14.990514 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990313    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-systemd-units\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.990514 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990339    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae6f9681-26d2-4340-8b43-fd6092388459-cert\") pod \"ingress-canary-xhrpk\" (UID: \"ae6f9681-26d2-4340-8b43-fd6092388459\") " pod="openshift-ingress-canary/ingress-canary-xhrpk"
Nov 04 03:17:14.990514 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990371    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-service-ca-bundle\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:14.990514 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990446    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdvfm\" (UniqueName: \"kubernetes.io/projected/7721e70f-2dda-400b-b1a7-d434249efb7e-kube-api-access-wdvfm\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:14.990514 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990490    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m65mn\" (UniqueName: \"kubernetes.io/projected/18329af9-cec7-48fd-aeee-18367bb6c078-kube-api-access-m65mn\") pod \"node-ca-tdgzf\" (UID: \"18329af9-cec7-48fd-aeee-18367bb6c078\") " pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:17:14.990514 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990518    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-cni-binary-copy\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:17:14.991123 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990543    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:17:14.991123 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990575    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-log-socket\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.991123 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990599    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-system-cni-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:14.991123 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990624    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovnkube-config\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.991123 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990650    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmbrz\" (UniqueName: \"kubernetes.io/projected/a4fa5e68-1a28-41d2-9680-c79f0d95e361-kube-api-access-vmbrz\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.991123 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990676    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmw2d\" (UniqueName: \"kubernetes.io/projected/ea8cb5af-3757-47db-840e-728d8d0ad62d-kube-api-access-dmw2d\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:17:14.991123 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990702    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e6f6098-4287-4076-a8c5-2e55067dfb87-konnectivity-ca\") pod \"konnectivity-agent-sd8pw\" (UID: \"7e6f6098-4287-4076-a8c5-2e55067dfb87\") " pod="kube-system/konnectivity-agent-sd8pw"
Nov 04 03:17:14.991123 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990729    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-systemd\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:14.991123 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990783    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-run\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:14.991123 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990816    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmghw\" (UniqueName: \"kubernetes.io/projected/70ed447c-add1-4e78-94fb-6aa3e4389fb5-kube-api-access-qmghw\") pod \"node-resolver-qk4nr\" (UID: \"70ed447c-add1-4e78-94fb-6aa3e4389fb5\") " pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:17:14.991123 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990880    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0924a641-d0e7-4cdb-8130-f17a47a4c026-kubelet-config\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:17:14.991123 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990943    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-trusted-ca-bundle\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:14.991123 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990972    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-etc-kubernetes\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:14.991123 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.990990    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-etc-selinux\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:14.991123 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991025    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96vcz\" (UniqueName: \"kubernetes.io/projected/4db0986b-dec5-4a04-b0f0-ebbc3298f250-kube-api-access-96vcz\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:14.991123 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991078    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a47538cd-f8b8-409f-8eed-c0d9881a822e-tmp-dir\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:14.991591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991150    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-cnibin\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:14.991591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991226    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysconfig\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:14.991591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991257    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysctl-conf\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:14.991591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991296    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs\") pod \"network-metrics-daemon-lgvq4\" (UID: \"23b77dc0-f7c4-4e72-a022-d75d8c8221fd\") " pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:17:14.991591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991333    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-slash\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.991591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991357    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/41f71514-1213-429b-8103-5fb88c9c7039-crio-socket\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:14.991591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991388    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-cni-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:14.991591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991417    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7721e70f-2dda-400b-b1a7-d434249efb7e-cni-binary-copy\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:14.991591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991436    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-cni-multus\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:14.991591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991479    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-kubernetes\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:14.991591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991523    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5d04c72f-9d65-429a-ad6c-dc69054ba75e-iptables-alerter-script\") pod \"iptables-alerter-m9zlr\" (UID: \"5d04c72f-9d65-429a-ad6c-dc69054ba75e\") " pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:17:14.991591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991546    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:14.991591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991565    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a47538cd-f8b8-409f-8eed-c0d9881a822e-tmp-dir\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:14.991950 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991610    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a47538cd-f8b8-409f-8eed-c0d9881a822e-config-volume\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:14.991950 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991671    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmgmq\" (UniqueName: \"kubernetes.io/projected/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-kube-api-access-bmgmq\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:14.991950 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:14.991692    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-cnibin\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:17:15.007328 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.007308    4521 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:17:15.007391 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:15.007361    4521 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:15.007391 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.007371    4521 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:17:15.007449 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:15.007404    4521 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.us-west-2.compute.internal"
Nov 04 03:17:15.026181 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.026160    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 03:17:15.026267 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.026235    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 03:17:15.026372 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.026347    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 03:17:15.026477 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.026437    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 03:17:15.026477 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.026469    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 03:17:15.026590 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.026525    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 03:17:15.026590 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.026551    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 03:17:15.026902 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.026881    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 03:17:15.026989 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.026902    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 03:17:15.026989 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.026926    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:17:15.026989 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.026962    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 03:17:15.027141 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.026994    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 03:17:15.027141 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.026999    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 03:17:15.027141 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.026996    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 03:17:15.027141 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.027006    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:17:15.027141 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.027049    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 03:17:15.027141 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.027078    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 03:17:15.027329 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.027153    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 03:17:15.027329 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.027295    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 03:17:15.027329 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.027302    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z2nst\""
Nov 04 03:17:15.027329 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.027295    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-569bn\""
Nov 04 03:17:15.027478 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.027335    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-zsv6g\""
Nov 04 03:17:15.027771 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.027343    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h66cx\""
Nov 04 03:17:15.028046 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.028017    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fbcnm\""
Nov 04 03:17:15.028120 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.027358    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-vpndz\""
Nov 04 03:17:15.028120 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.027355    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xwzp4\""
Nov 04 03:17:15.028219 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.028169    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 03:17:15.028219 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.028197    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4vrb5\""
Nov 04 03:17:15.029897 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.029879    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-mnqmt\""
Nov 04 03:17:15.030190 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.030171    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e6f6098-4287-4076-a8c5-2e55067dfb87-agent-certs\") pod \"konnectivity-agent-sd8pw\" (UID: \"7e6f6098-4287-4076-a8c5-2e55067dfb87\") " pod="kube-system/konnectivity-agent-sd8pw"
Nov 04 03:17:15.030293 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.030223    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0924a641-d0e7-4cdb-8130-f17a47a4c026-original-pull-secret\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:17:15.030423 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.030403    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/18329af9-cec7-48fd-aeee-18367bb6c078-serviceca\") pod \"node-ca-tdgzf\" (UID: \"18329af9-cec7-48fd-aeee-18367bb6c078\") " pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:17:15.030514 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.030459    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-daemon-config\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.031167 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.031066    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e6f6098-4287-4076-a8c5-2e55067dfb87-konnectivity-ca\") pod \"konnectivity-agent-sd8pw\" (UID: \"7e6f6098-4287-4076-a8c5-2e55067dfb87\") " pod="kube-system/konnectivity-agent-sd8pw"
Nov 04 03:17:15.031167 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.031097    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-cni-binary-copy\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:17:15.032058 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.032042    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7721e70f-2dda-400b-b1a7-d434249efb7e-cni-binary-copy\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.032058 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.032050    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5d04c72f-9d65-429a-ad6c-dc69054ba75e-iptables-alerter-script\") pod \"iptables-alerter-m9zlr\" (UID: \"5d04c72f-9d65-429a-ad6c-dc69054ba75e\") " pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:17:15.049819 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.049796    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x6xxb\""
Nov 04 03:17:15.050139 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.050123    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9cxzk\""
Nov 04 03:17:15.050249 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.050231    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 03:17:15.050984 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.050969    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-56wj9\""
Nov 04 03:17:15.051341 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.051323    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-mhdb8\""
Nov 04 03:17:15.051681 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.051657    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 03:17:15.051799 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.051690    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 03:17:15.051799 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.051701    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 03:17:15.051799 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.051705    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vjndb\""
Nov 04 03:17:15.057726 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.057713    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 03:17:15.057815 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.057732    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-gj69c\""
Nov 04 03:17:15.057899 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.057888    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 04 03:17:15.061251 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.061236    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovn-node-metrics-cert\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.061398 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.061382    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae6f9681-26d2-4340-8b43-fd6092388459-cert\") pod \"ingress-canary-xhrpk\" (UID: \"ae6f9681-26d2-4340-8b43-fd6092388459\") " pod="openshift-ingress-canary/ingress-canary-xhrpk"
Nov 04 03:17:15.075416 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.075393    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5zf5b\""
Nov 04 03:17:15.075731 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.075708    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-xs24w\""
Nov 04 03:17:15.075813 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.075786    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jw2mq\""
Nov 04 03:17:15.076045 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.076028    4521 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 03:17:15.076348 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.076329    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmghw\" (UniqueName: \"kubernetes.io/projected/70ed447c-add1-4e78-94fb-6aa3e4389fb5-kube-api-access-qmghw\") pod \"node-resolver-qk4nr\" (UID: \"70ed447c-add1-4e78-94fb-6aa3e4389fb5\") " pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:17:15.076385 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.076329    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nnm8\" (UniqueName: \"kubernetes.io/projected/16976140-0bb0-4983-bcb7-b2fa2274d24e-kube-api-access-7nnm8\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.076423 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.076407    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgp4k\" (UniqueName: \"kubernetes.io/projected/b7f9ded5-2460-4131-ab22-85c462aaae48-kube-api-access-hgp4k\") pod \"network-check-target-6kpt2\" (UID: \"b7f9ded5-2460-4131-ab22-85c462aaae48\") " pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:17:15.076948 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.076903    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kmrh\" (UniqueName: \"kubernetes.io/projected/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-kube-api-access-7kmrh\") pod \"network-metrics-daemon-lgvq4\" (UID: \"23b77dc0-f7c4-4e72-a022-d75d8c8221fd\") " pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:17:15.077068 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.077050    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m65mn\" (UniqueName: \"kubernetes.io/projected/18329af9-cec7-48fd-aeee-18367bb6c078-kube-api-access-m65mn\") pod \"node-ca-tdgzf\" (UID: \"18329af9-cec7-48fd-aeee-18367bb6c078\") " pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:17:15.077144 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.077088    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmw2d\" (UniqueName: \"kubernetes.io/projected/ea8cb5af-3757-47db-840e-728d8d0ad62d-kube-api-access-dmw2d\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:17:15.077144 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.077106    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq5wt\" (UniqueName: \"kubernetes.io/projected/5d04c72f-9d65-429a-ad6c-dc69054ba75e-kube-api-access-pq5wt\") pod \"iptables-alerter-m9zlr\" (UID: \"5d04c72f-9d65-429a-ad6c-dc69054ba75e\") " pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:17:15.077246 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.077164    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86ljh\" (UniqueName: \"kubernetes.io/projected/a47538cd-f8b8-409f-8eed-c0d9881a822e-kube-api-access-86ljh\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:15.077246 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.077195    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdvfm\" (UniqueName: \"kubernetes.io/projected/7721e70f-2dda-400b-b1a7-d434249efb7e-kube-api-access-wdvfm\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.092133 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092111    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70ed447c-add1-4e78-94fb-6aa3e4389fb5-hosts-file\") pod \"node-resolver-qk4nr\" (UID: \"70ed447c-add1-4e78-94fb-6aa3e4389fb5\") " pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:17:15.092215 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092135    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-ovn\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.092215 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092151    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-modprobe-d\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.092215 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092164    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0924a641-d0e7-4cdb-8130-f17a47a4c026-dbus\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:17:15.092215 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092179    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-node-log\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.092371 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092210    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-cni-bin\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.092371 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092244    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0924a641-d0e7-4cdb-8130-f17a47a4c026-dbus\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:17:15.092371 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092252    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d04c72f-9d65-429a-ad6c-dc69054ba75e-host-slash\") pod \"iptables-alerter-m9zlr\" (UID: \"5d04c72f-9d65-429a-ad6c-dc69054ba75e\") " pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:17:15.092371 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092216    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-ovn\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.092371 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092252    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-cni-bin\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.092371 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092274    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70ed447c-add1-4e78-94fb-6aa3e4389fb5-hosts-file\") pod \"node-resolver-qk4nr\" (UID: \"70ed447c-add1-4e78-94fb-6aa3e4389fb5\") " pod="openshift-dns/node-resolver-qk4nr"
Nov 04 03:17:15.092371 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092280    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-sys-fs\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:15.092371 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092284    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-modprobe-d\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.092371 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092242    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-node-log\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.092371 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092293    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d04c72f-9d65-429a-ad6c-dc69054ba75e-host-slash\") pod \"iptables-alerter-m9zlr\" (UID: \"5d04c72f-9d65-429a-ad6c-dc69054ba75e\") " pod="openshift-network-operator/iptables-alerter-m9zlr"
Nov 04 03:17:15.092371 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092325    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-sys-fs\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:15.092371 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092336    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-etc-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.092371 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092366    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-etc-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.092371 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092375    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-lib-modules\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092405    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-var-lib-kubelet\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092436    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-var-lib-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092443    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-lib-modules\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092455    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-var-lib-kubelet\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092456    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092475    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-var-lib-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092482    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-socket-dir-parent\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092505    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-systemd-units\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092523    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-log-socket\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092537    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-system-cni-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092557    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98vqg\" (UniqueName: \"kubernetes.io/projected/62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8-kube-api-access-98vqg\") pod \"migrator-66df5d496b-9fzvp\" (UID: \"62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9fzvp"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092562    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-systemd-units\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092590    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-log-socket\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092596    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-systemd\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092590    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-socket-dir-parent\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092611    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-system-cni-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092619    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-run\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092644    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0924a641-d0e7-4cdb-8130-f17a47a4c026-kubelet-config\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092651    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-systemd\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092670    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-etc-kubernetes\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092685    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-run\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092695    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-etc-selinux\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092701    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0924a641-d0e7-4cdb-8130-f17a47a4c026-kubelet-config\") pod \"global-pull-secret-syncer-qbcj2\" (UID: \"0924a641-d0e7-4cdb-8130-f17a47a4c026\") " pod="kube-system/global-pull-secret-syncer-qbcj2"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092712    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-etc-kubernetes\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092720    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xts4\" (UniqueName: \"kubernetes.io/projected/ab11857e-7c79-464b-aba7-c6df7ef55a7f-kube-api-access-9xts4\") pod \"service-ca-5fffc97bdf-pnj7z\" (UID: \"ab11857e-7c79-464b-aba7-c6df7ef55a7f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092746    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-cnibin\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092754    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-etc-selinux\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092770    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysconfig\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.092783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092795    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysctl-conf\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092805    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-cnibin\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092821    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ab11857e-7c79-464b-aba7-c6df7ef55a7f-signing-cabundle\") pod \"service-ca-5fffc97bdf-pnj7z\" (UID: \"ab11857e-7c79-464b-aba7-c6df7ef55a7f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092839    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs\") pod \"network-metrics-daemon-lgvq4\" (UID: \"23b77dc0-f7c4-4e72-a022-d75d8c8221fd\") " pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092873    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-slash\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092889    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/41f71514-1213-429b-8103-5fb88c9c7039-crio-socket\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092903    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-cni-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092918    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-cni-multus\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092935    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-kubernetes\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092959    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092963    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysctl-conf\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092975    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysconfig\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092985    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-cnibin\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092988    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-cni-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.092975    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/41f71514-1213-429b-8103-5fb88c9c7039-crio-socket\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093021    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-openvswitch\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093023    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-cni-multus\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093031    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-cnibin\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093020    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-slash\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093022    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-kubernetes\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093047    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093072    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093093    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ab11857e-7c79-464b-aba7-c6df7ef55a7f-signing-key\") pod \"service-ca-5fffc97bdf-pnj7z\" (UID: \"ab11857e-7c79-464b-aba7-c6df7ef55a7f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093125    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-run-ovn-kubernetes\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093149    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysctl-d\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093173    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-sys\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093198    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-run-ovn-kubernetes\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093200    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-host\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093229    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-sys\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093253    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-host\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093244    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-cni-bin\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093273    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-cni-bin\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093227    4521 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093281    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/16976140-0bb0-4983-bcb7-b2fa2274d24e-etc-sysctl-d\") pod \"tuned-bkcnt\" (UID: \"16976140-0bb0-4983-bcb7-b2fa2274d24e\") " pod="openshift-cluster-node-tuning-operator/tuned-bkcnt"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093300    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ab11857e-7c79-464b-aba7-c6df7ef55a7f-signing-key\") pod \"service-ca-5fffc97bdf-pnj7z\" (UID: \"ab11857e-7c79-464b-aba7-c6df7ef55a7f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093300    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-kubelet\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093321    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-var-lib-kubelet\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093342    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-hostroot\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093361    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-multus-certs\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093377    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-systemd\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093381    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-hostroot\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093393    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-cni-netd\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093403    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-multus-certs\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093410    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-run-systemd\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093415    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093433    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-netns\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093460    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-conf-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093472    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-netns\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093484    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093435    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-cni-netd\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:15.093483    4521 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093510    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-multus-conf-dir\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093516    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18329af9-cec7-48fd-aeee-18367bb6c078-host\") pod \"node-ca-tdgzf\" (UID: \"18329af9-cec7-48fd-aeee-18367bb6c078\") " pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093544    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-kubelet\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093567    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18329af9-cec7-48fd-aeee-18367bb6c078-host\") pod \"node-ca-tdgzf\" (UID: \"18329af9-cec7-48fd-aeee-18367bb6c078\") " pod="openshift-image-registry/node-ca-tdgzf"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093570    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-kubelet\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093569    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093607    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:15.093619    4521 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:15.593582596 +0000 UTC m=+1.798182876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093652    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-system-cni-dir\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093680    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093704    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-socket-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093718    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8cb5af-3757-47db-840e-728d8d0ad62d-system-cni-dir\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093729    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-registration-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093749    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093754    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-device-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093776    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-socket-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093790    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-registration-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093778    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-run-netns\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093810    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a4fa5e68-1a28-41d2-9680-c79f0d95e361-host-run-netns\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093825    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4db0986b-dec5-4a04-b0f0-ebbc3298f250-device-dir\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093830    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-k8s-cni-cncf-io\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.093891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.093881    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7721e70f-2dda-400b-b1a7-d434249efb7e-host-run-k8s-cni-cncf-io\") pod \"multus-hwrqp\" (UID: \"7721e70f-2dda-400b-b1a7-d434249efb7e\") " pod="openshift-multus/multus-hwrqp"
Nov 04 03:17:15.096008 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.095468    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23b77dc0-f7c4-4e72-a022-d75d8c8221fd-metrics-certs\") pod \"network-metrics-daemon-lgvq4\" (UID: \"23b77dc0-f7c4-4e72-a022-d75d8c8221fd\") " pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:17:15.096008 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.095525    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a47538cd-f8b8-409f-8eed-c0d9881a822e-metrics-tls\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:15.126967 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.126912    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 03:17:15.127084 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127069    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 03:17:15.127193 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127176    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 03:17:15.127260 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127182    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 03:17:15.127260 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127209    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 04 03:17:15.127260 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127229    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 03:17:15.127542 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127524    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 03:17:15.127542 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127538    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 04 03:17:15.127949 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127569    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 03:17:15.127949 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127600    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 04 03:17:15.127949 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127606    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 03:17:15.127949 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127631    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 03:17:15.127949 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127604    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 04 03:17:15.127949 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127651    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 04 03:17:15.127949 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127669    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 03:17:15.127949 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127678    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 04 03:17:15.127949 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127660    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 03:17:15.127949 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127699    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 03:17:15.127949 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.127732    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 03:17:15.128303 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.128021    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 03:17:15.128303 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.128044    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 03:17:15.128375 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.128360    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 04 03:17:15.128502 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.128487    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:17:15.129347 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.129332    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-env-overrides\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.130523 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.130510    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-service-ca-bundle\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:15.130523 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.130515    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovnkube-script-lib\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.131706 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.131683    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ea8cb5af-3757-47db-840e-728d8d0ad62d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xrrqq\" (UID: \"ea8cb5af-3757-47db-840e-728d8d0ad62d\") " pod="openshift-multus/multus-additional-cni-plugins-xrrqq"
Nov 04 03:17:15.131745 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.131713    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a4fa5e68-1a28-41d2-9680-c79f0d95e361-ovnkube-config\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.131773 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.131745    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-trusted-ca-bundle\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:15.131802 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.131793    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a47538cd-f8b8-409f-8eed-c0d9881a822e-config-volume\") pod \"dns-default-vwf9m\" (UID: \"a47538cd-f8b8-409f-8eed-c0d9881a822e\") " pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:15.132775 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:15.132761    4521 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:15.132829 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:15.132820    4521 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:15.632807426 +0000 UTC m=+1.837407706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:15.133973 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.133960    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ab11857e-7c79-464b-aba7-c6df7ef55a7f-signing-cabundle\") pod \"service-ca-5fffc97bdf-pnj7z\" (UID: \"ab11857e-7c79-464b-aba7-c6df7ef55a7f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z"
Nov 04 03:17:15.136360 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.136345    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jwdt\" (UniqueName: \"kubernetes.io/projected/41f71514-1213-429b-8103-5fb88c9c7039-kube-api-access-7jwdt\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:15.136443 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.136429    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmbrz\" (UniqueName: \"kubernetes.io/projected/a4fa5e68-1a28-41d2-9680-c79f0d95e361-kube-api-access-vmbrz\") pod \"ovnkube-node-zfmg5\" (UID: \"a4fa5e68-1a28-41d2-9680-c79f0d95e361\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:15.136542 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.136526    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvrhn\" (UniqueName: \"kubernetes.io/projected/ae6f9681-26d2-4340-8b43-fd6092388459-kube-api-access-jvrhn\") pod \"ingress-canary-xhrpk\" (UID: \"ae6f9681-26d2-4340-8b43-fd6092388459\") " pod="openshift-ingress-canary/ingress-canary-xhrpk"
Nov 04 03:17:15.137202 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.137187    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmgmq\" (UniqueName: \"kubernetes.io/projected/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-kube-api-access-bmgmq\") pod \"periodic-gathering-dm5lv-xwxch\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") " pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:17:15.137239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.137207    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96vcz\" (UniqueName: \"kubernetes.io/projected/4db0986b-dec5-4a04-b0f0-ebbc3298f250-kube-api-access-96vcz\") pod \"aws-ebs-csi-driver-node-k82g4\" (UID: \"4db0986b-dec5-4a04-b0f0-ebbc3298f250\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k82g4"
Nov 04 03:17:15.151623 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.151600    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98vqg\" (UniqueName: \"kubernetes.io/projected/62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8-kube-api-access-98vqg\") pod \"migrator-66df5d496b-9fzvp\" (UID: \"62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9fzvp"
Nov 04 03:17:15.151705 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.151690    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xts4\" (UniqueName: \"kubernetes.io/projected/ab11857e-7c79-464b-aba7-c6df7ef55a7f-kube-api-access-9xts4\") pod \"service-ca-5fffc97bdf-pnj7z\" (UID: \"ab11857e-7c79-464b-aba7-c6df7ef55a7f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pnj7z"
Nov 04 03:17:15.273662 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.273636    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lgvq4"
Nov 04 03:17:15.275886 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.275861    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:15.541070 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.541044    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xhrpk_ae6f9681-26d2-4340-8b43-fd6092388459/serve-healthcheck-canary/0.log"
Nov 04 03:17:15.598720 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.598690    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:15.598879 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:15.598790    4521 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:15.598879 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:15.598839    4521 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:16.598827093 +0000 UTC m=+2.803427373 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:15.651456 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.651347    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lgvq4"]
Nov 04 03:17:15.655972 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:15.655940    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23b77dc0_f7c4_4e72_a022_d75d8c8221fd.slice/crio-a8e844c9e4747702e8b7c8e5cdcc335d69719beaf69cc5b868466d403175eed4 WatchSource:0}: Error finding container a8e844c9e4747702e8b7c8e5cdcc335d69719beaf69cc5b868466d403175eed4: Status 404 returned error can't find the container with id a8e844c9e4747702e8b7c8e5cdcc335d69719beaf69cc5b868466d403175eed4
Nov 04 03:17:15.657978 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.657957    4521 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:17:15.699452 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.699420    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:15.699575 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:15.699499    4521 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:15.699575 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:15.699545    4521 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:16.699533386 +0000 UTC m=+2.904133666 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:15.702446 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.702419    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vwf9m"]
Nov 04 03:17:15.706200 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:15.706177    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda47538cd_f8b8_409f_8eed_c0d9881a822e.slice/crio-120cce8ec90ade0394026a489fb4f0bf61224b11f4baa0caaa09fdf4591faab1 WatchSource:0}: Error finding container 120cce8ec90ade0394026a489fb4f0bf61224b11f4baa0caaa09fdf4591faab1: Status 404 returned error can't find the container with id 120cce8ec90ade0394026a489fb4f0bf61224b11f4baa0caaa09fdf4591faab1
Nov 04 03:17:15.740505 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.740471    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-9fzvp_62b0b5f0-3fb2-4d16-9017-3ae27a0e84e8/migrator/0.log"
Nov 04 03:17:15.948954 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.948873    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vwf9m" event={"ID":"a47538cd-f8b8-409f-8eed-c0d9881a822e","Type":"ContainerStarted","Data":"120cce8ec90ade0394026a489fb4f0bf61224b11f4baa0caaa09fdf4591faab1"}
Nov 04 03:17:15.950039 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:15.950008    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lgvq4" event={"ID":"23b77dc0-f7c4-4e72-a022-d75d8c8221fd","Type":"ContainerStarted","Data":"a8e844c9e4747702e8b7c8e5cdcc335d69719beaf69cc5b868466d403175eed4"}
Nov 04 03:17:16.604110 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:16.604074    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:16.604274 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:16.604189    4521 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:16.604274 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:16.604250    4521 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:18.604237562 +0000 UTC m=+4.808837855 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:16.704765 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:16.704735    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:16.704924 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:16.704822    4521 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:16.704924 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:16.704892    4521 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:18.704879943 +0000 UTC m=+4.909480223 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:17.962746 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:17.962693    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lgvq4" event={"ID":"23b77dc0-f7c4-4e72-a022-d75d8c8221fd","Type":"ContainerStarted","Data":"c547dd47a887cf1a7c2c4d977f2ab0f9e16fab66c531c5d2dc3eaaa9cabbc927"}
Nov 04 03:17:17.963120 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:17.962787    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lgvq4" event={"ID":"23b77dc0-f7c4-4e72-a022-d75d8c8221fd","Type":"ContainerStarted","Data":"4279043e638c4535fc9950908bf4bb7912654050380fd0507308976838145302"}
Nov 04 03:17:18.619818 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:18.619781    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:18.619998 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:18.619942    4521 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:18.620042 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:18.620008    4521 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:22.619992416 +0000 UTC m=+8.824592696 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:18.720894 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:18.720864    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:18.721041 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:18.720955    4521 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:18.721041 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:18.721003    4521 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:22.7209916 +0000 UTC m=+8.925591880 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:18.967824 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:18.967738    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vwf9m" event={"ID":"a47538cd-f8b8-409f-8eed-c0d9881a822e","Type":"ContainerStarted","Data":"54dc71d97bef4a4eb1614bebc84816110ffa5eaaddd9df403b11b090988e7825"}
Nov 04 03:17:18.967824 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:18.967792    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vwf9m" event={"ID":"a47538cd-f8b8-409f-8eed-c0d9881a822e","Type":"ContainerStarted","Data":"960e900da8a8a25144379aae75bb37a760d1a2ddf00e8665156ac27c02eceaf5"}
Nov 04 03:17:19.971178 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:19.971145    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:21.858667 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:21.858623    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:17:22.645602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:22.645562    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:22.645755 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:22.645685    4521 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:22.645755 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:22.645742    4521 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls podName:41f71514-1213-429b-8103-5fb88c9c7039 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:30.645727862 +0000 UTC m=+16.850328142 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5zsk" (UID: "41f71514-1213-429b-8103-5fb88c9c7039") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:22.746637 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:22.746601    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:22.747133 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:22.747113    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f71514-1213-429b-8103-5fb88c9c7039-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:23.467610 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:23.467583    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:24.087832 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:24.087807    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:24.088065 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:24.088048    4521 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 03:17:24.660928 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:24.660896    4521 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-242.us-west-2.compute.internal" event="NodeReady"
Nov 04 03:17:25.051043 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:25.051017    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfmg5"
Nov 04 03:17:29.977989 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:29.977963    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vwf9m"
Nov 04 03:17:30.700365 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:30.700329    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:30.702747 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:30.702719    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f71514-1213-429b-8103-5fb88c9c7039-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5zsk\" (UID: \"41f71514-1213-429b-8103-5fb88c9c7039\") " pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:30.876901 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:30.876864    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t5zsk"
Nov 04 03:17:31.239934 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:31.239901    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sd8pw"
Nov 04 03:17:31.264967 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:31.264942    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t5zsk"]
Nov 04 03:17:31.268601 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:31.268574    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41f71514_1213_429b_8103_5fb88c9c7039.slice/crio-4630e5c793fb4748d2a0c4c4f5fc13a94e3fec4cc0cce22d7f033258a7578502 WatchSource:0}: Error finding container 4630e5c793fb4748d2a0c4c4f5fc13a94e3fec4cc0cce22d7f033258a7578502: Status 404 returned error can't find the container with id 4630e5c793fb4748d2a0c4c4f5fc13a94e3fec4cc0cce22d7f033258a7578502
Nov 04 03:17:32.005667 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:32.005633    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t5zsk" event={"ID":"41f71514-1213-429b-8103-5fb88c9c7039","Type":"ContainerStarted","Data":"c556d9ee953fc435fd632366593db2e77a1a2092ffadc09a026ba206a44d0928"}
Nov 04 03:17:32.005829 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:32.005690    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t5zsk" event={"ID":"41f71514-1213-429b-8103-5fb88c9c7039","Type":"ContainerStarted","Data":"4630e5c793fb4748d2a0c4c4f5fc13a94e3fec4cc0cce22d7f033258a7578502"}
Nov 04 03:17:34.012626 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:34.012593    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t5zsk" event={"ID":"41f71514-1213-429b-8103-5fb88c9c7039","Type":"ContainerStarted","Data":"1960b998c2a4c5df4b38c26146590da83dc680c517a69c07d3f21ce268f3399e"}
Nov 04 03:17:36.021068 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:36.020983    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t5zsk" event={"ID":"41f71514-1213-429b-8103-5fb88c9c7039","Type":"ContainerStarted","Data":"db67a9b1da62edc9fbae2d70c3b52d47a7961f682c1cf624300e22a53e91f5cd"}
Nov 04 03:17:36.537909 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:36.537876    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hw6vw"]
Nov 04 03:17:36.559292 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:36.559268    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hw6vw"
Nov 04 03:17:36.599257 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:36.599231    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hw6vw"]
Nov 04 03:17:36.612414 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:36.612394    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Nov 04 03:17:36.612539 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:36.612458    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-9dqw6\""
Nov 04 03:17:36.643769 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:36.643731    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/50350caf-69d6-47f8-9e21-c19439ce983e-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-hw6vw\" (UID: \"50350caf-69d6-47f8-9e21-c19439ce983e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hw6vw"
Nov 04 03:17:36.745079 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:36.745046    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/50350caf-69d6-47f8-9e21-c19439ce983e-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-hw6vw\" (UID: \"50350caf-69d6-47f8-9e21-c19439ce983e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hw6vw"
Nov 04 03:17:36.747541 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:36.747514    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/50350caf-69d6-47f8-9e21-c19439ce983e-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-hw6vw\" (UID: \"50350caf-69d6-47f8-9e21-c19439ce983e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hw6vw"
Nov 04 03:17:36.884039 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:36.883952    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hw6vw"
Nov 04 03:17:37.250796 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:37.250715    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hw6vw"]
Nov 04 03:17:37.254128 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:37.254091    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50350caf_69d6_47f8_9e21_c19439ce983e.slice/crio-7c4361eb4a8c9fc2192a421fb65f21dc0da56504a028fe4989dd540d7415de1c WatchSource:0}: Error finding container 7c4361eb4a8c9fc2192a421fb65f21dc0da56504a028fe4989dd540d7415de1c: Status 404 returned error can't find the container with id 7c4361eb4a8c9fc2192a421fb65f21dc0da56504a028fe4989dd540d7415de1c
Nov 04 03:17:38.028609 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:38.028568    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hw6vw" event={"ID":"50350caf-69d6-47f8-9e21-c19439ce983e","Type":"ContainerStarted","Data":"7c4361eb4a8c9fc2192a421fb65f21dc0da56504a028fe4989dd540d7415de1c"}
Nov 04 03:17:39.033138 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:39.033103    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hw6vw" event={"ID":"50350caf-69d6-47f8-9e21-c19439ce983e","Type":"ContainerStarted","Data":"a52f3b2bca0553ff77be2ed596b42c8ba9995e2f9fb7b7456f1630561e94c3f2"}
Nov 04 03:17:39.033606 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:39.033265    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hw6vw"
Nov 04 03:17:39.037976 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:39.037954    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hw6vw"
Nov 04 03:17:39.149050 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:39.148934    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hw6vw" podStartSLOduration=1.5695927809999999 podStartE2EDuration="3.148920756s" totalImagesPullingTime="1.579327975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:39.148704602 +0000 UTC m=+25.353304919" watchObservedRunningTime="2026-11-04 03:17:39.148920756 +0000 UTC m=+25.353521058"
Nov 04 03:17:40.157195 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.157161    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-67vg7"]
Nov 04 03:17:40.159422 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.159404    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7"
Nov 04 03:17:40.263026 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.262992    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-ps95k\""
Nov 04 03:17:40.263026 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.262992    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Nov 04 03:17:40.263204 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.263069    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Nov 04 03:17:40.263482 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.263462    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-67vg7"]
Nov 04 03:17:40.268426 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.268086    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/de7361e1-276a-489b-bfb8-140e94f3dd57-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-67vg7\" (UID: \"de7361e1-276a-489b-bfb8-140e94f3dd57\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7"
Nov 04 03:17:40.268426 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.268130    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc8s4\" (UniqueName: \"kubernetes.io/projected/de7361e1-276a-489b-bfb8-140e94f3dd57-kube-api-access-cc8s4\") pod \"prometheus-operator-9cd89b74c-67vg7\" (UID: \"de7361e1-276a-489b-bfb8-140e94f3dd57\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7"
Nov 04 03:17:40.268426 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.268185    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de7361e1-276a-489b-bfb8-140e94f3dd57-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-67vg7\" (UID: \"de7361e1-276a-489b-bfb8-140e94f3dd57\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7"
Nov 04 03:17:40.268426 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.268220    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de7361e1-276a-489b-bfb8-140e94f3dd57-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-67vg7\" (UID: \"de7361e1-276a-489b-bfb8-140e94f3dd57\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7"
Nov 04 03:17:40.269259 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.269232    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 03:17:40.269370 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.269349    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 03:17:40.313542 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.313516    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 04 03:17:40.369046 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.369014    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/de7361e1-276a-489b-bfb8-140e94f3dd57-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-67vg7\" (UID: \"de7361e1-276a-489b-bfb8-140e94f3dd57\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7"
Nov 04 03:17:40.369046 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.369049    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc8s4\" (UniqueName: \"kubernetes.io/projected/de7361e1-276a-489b-bfb8-140e94f3dd57-kube-api-access-cc8s4\") pod \"prometheus-operator-9cd89b74c-67vg7\" (UID: \"de7361e1-276a-489b-bfb8-140e94f3dd57\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7"
Nov 04 03:17:40.369254 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.369080    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de7361e1-276a-489b-bfb8-140e94f3dd57-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-67vg7\" (UID: \"de7361e1-276a-489b-bfb8-140e94f3dd57\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7"
Nov 04 03:17:40.369254 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.369104    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de7361e1-276a-489b-bfb8-140e94f3dd57-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-67vg7\" (UID: \"de7361e1-276a-489b-bfb8-140e94f3dd57\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7"
Nov 04 03:17:40.369716 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.369695    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de7361e1-276a-489b-bfb8-140e94f3dd57-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-67vg7\" (UID: \"de7361e1-276a-489b-bfb8-140e94f3dd57\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7"
Nov 04 03:17:40.371465 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.371445    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de7361e1-276a-489b-bfb8-140e94f3dd57-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-67vg7\" (UID: \"de7361e1-276a-489b-bfb8-140e94f3dd57\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7"
Nov 04 03:17:40.371465 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.371457    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/de7361e1-276a-489b-bfb8-140e94f3dd57-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-67vg7\" (UID: \"de7361e1-276a-489b-bfb8-140e94f3dd57\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7"
Nov 04 03:17:40.436737 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.436662    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc8s4\" (UniqueName: \"kubernetes.io/projected/de7361e1-276a-489b-bfb8-140e94f3dd57-kube-api-access-cc8s4\") pod \"prometheus-operator-9cd89b74c-67vg7\" (UID: \"de7361e1-276a-489b-bfb8-140e94f3dd57\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7"
Nov 04 03:17:40.470283 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.470251    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7"
Nov 04 03:17:40.837685 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:40.837650    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-67vg7"]
Nov 04 03:17:40.840941 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:40.840911    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde7361e1_276a_489b_bfb8_140e94f3dd57.slice/crio-81eb7e84858b79b2ea4ac3f73bec2541e2648c26e386c0da2aad73a7a905eb94 WatchSource:0}: Error finding container 81eb7e84858b79b2ea4ac3f73bec2541e2648c26e386c0da2aad73a7a905eb94: Status 404 returned error can't find the container with id 81eb7e84858b79b2ea4ac3f73bec2541e2648c26e386c0da2aad73a7a905eb94
Nov 04 03:17:41.041907 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:41.041870    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7" event={"ID":"de7361e1-276a-489b-bfb8-140e94f3dd57","Type":"ContainerStarted","Data":"81eb7e84858b79b2ea4ac3f73bec2541e2648c26e386c0da2aad73a7a905eb94"}
Nov 04 03:17:41.861784 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:41.861754    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6kpt2"
Nov 04 03:17:44.055137 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:44.055096    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7" event={"ID":"de7361e1-276a-489b-bfb8-140e94f3dd57","Type":"ContainerStarted","Data":"15c4a817190715fc85292cc25b15875308a0ee13fdfc16c4ee0a6a4579c6978a"}
Nov 04 03:17:44.055613 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:44.055163    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7" event={"ID":"de7361e1-276a-489b-bfb8-140e94f3dd57","Type":"ContainerStarted","Data":"c313e692f53f656e0a1890ab863a584912bc23ff2942d17110b5da9a41b0b655"}
Nov 04 03:17:44.173135 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:44.173078    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-67vg7" podStartSLOduration=2.009653611 podStartE2EDuration="4.173061688s" totalImagesPullingTime="2.163408077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:44.172277105 +0000 UTC m=+30.376877406" watchObservedRunningTime="2026-11-04 03:17:44.173061688 +0000 UTC m=+30.377661991"
Nov 04 03:17:47.988305 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:47.988263    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xng65"]
Nov 04 03:17:47.991004 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:47.990980    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.113756 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.113723    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 04 03:17:48.114198 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.114176    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 04 03:17:48.114302 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.114286    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 04 03:17:48.114337 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.114294    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ctwv8\""
Nov 04 03:17:48.124620 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.124590    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b3f8b142-e183-45b1-a9b8-1581a885b0c0-root\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.124736 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.124627    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nspqf\" (UniqueName: \"kubernetes.io/projected/b3f8b142-e183-45b1-a9b8-1581a885b0c0-kube-api-access-nspqf\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.124736 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.124654    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b3f8b142-e183-45b1-a9b8-1581a885b0c0-node-exporter-wtmp\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.124736 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.124723    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b3f8b142-e183-45b1-a9b8-1581a885b0c0-node-exporter-textfile\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.124830 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.124756    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3f8b142-e183-45b1-a9b8-1581a885b0c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.124888 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.124820    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3f8b142-e183-45b1-a9b8-1581a885b0c0-sys\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.124958 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.124941    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3f8b142-e183-45b1-a9b8-1581a885b0c0-metrics-client-ca\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.125009 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.124994    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b3f8b142-e183-45b1-a9b8-1581a885b0c0-node-exporter-tls\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.125042 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.125018    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b3f8b142-e183-45b1-a9b8-1581a885b0c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.225746 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.225711    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b3f8b142-e183-45b1-a9b8-1581a885b0c0-root\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.225746 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.225748    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nspqf\" (UniqueName: \"kubernetes.io/projected/b3f8b142-e183-45b1-a9b8-1581a885b0c0-kube-api-access-nspqf\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.226152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.225768    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b3f8b142-e183-45b1-a9b8-1581a885b0c0-node-exporter-wtmp\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.226152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.225787    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b3f8b142-e183-45b1-a9b8-1581a885b0c0-node-exporter-textfile\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.226152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.225810    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3f8b142-e183-45b1-a9b8-1581a885b0c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.226152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.225833    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b3f8b142-e183-45b1-a9b8-1581a885b0c0-root\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.226152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.225864    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3f8b142-e183-45b1-a9b8-1581a885b0c0-sys\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.226152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.225909    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3f8b142-e183-45b1-a9b8-1581a885b0c0-sys\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.226152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.225924    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3f8b142-e183-45b1-a9b8-1581a885b0c0-metrics-client-ca\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.226152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.225959    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b3f8b142-e183-45b1-a9b8-1581a885b0c0-node-exporter-tls\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.226152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.225942    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b3f8b142-e183-45b1-a9b8-1581a885b0c0-node-exporter-wtmp\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.226152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.225989    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b3f8b142-e183-45b1-a9b8-1581a885b0c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.226557 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.226189    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b3f8b142-e183-45b1-a9b8-1581a885b0c0-node-exporter-textfile\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.226557 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.226479    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b3f8b142-e183-45b1-a9b8-1581a885b0c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.226664 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.226643    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3f8b142-e183-45b1-a9b8-1581a885b0c0-metrics-client-ca\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.228182 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.228157    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b3f8b142-e183-45b1-a9b8-1581a885b0c0-node-exporter-tls\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.228304 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.228282    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3f8b142-e183-45b1-a9b8-1581a885b0c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.286907 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.286874    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nspqf\" (UniqueName: \"kubernetes.io/projected/b3f8b142-e183-45b1-a9b8-1581a885b0c0-kube-api-access-nspqf\") pod \"node-exporter-xng65\" (UID: \"b3f8b142-e183-45b1-a9b8-1581a885b0c0\") " pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.303800 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:48.303775    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xng65"
Nov 04 03:17:48.314194 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:48.314167    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3f8b142_e183_45b1_a9b8_1581a885b0c0.slice/crio-8cc63bb1ca5e647308cbe15350e1239d434c227ed370e05a5a482d4494b2e547 WatchSource:0}: Error finding container 8cc63bb1ca5e647308cbe15350e1239d434c227ed370e05a5a482d4494b2e547: Status 404 returned error can't find the container with id 8cc63bb1ca5e647308cbe15350e1239d434c227ed370e05a5a482d4494b2e547
Nov 04 03:17:49.073643 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:49.073601    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xng65" event={"ID":"b3f8b142-e183-45b1-a9b8-1581a885b0c0","Type":"ContainerStarted","Data":"8cc63bb1ca5e647308cbe15350e1239d434c227ed370e05a5a482d4494b2e547"}
Nov 04 03:17:50.302536 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.302479    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:17:50.305276 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.305254    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.355670 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.355643    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:17:50.361020 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.360994    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 04 03:17:50.361127 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.361039    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 04 03:17:50.404725 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.404679    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 04 03:17:50.404725 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.404703    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 04 03:17:50.405023 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.404748    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 04 03:17:50.405023 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.404814    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 04 03:17:50.405023 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.404839    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 04 03:17:50.405023 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.404867    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 04 03:17:50.405171 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.405030    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-c24br\""
Nov 04 03:17:50.441957 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.441887    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.441957 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.441928    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5139d6b-153a-4ba1-a0be-17cac7482f92-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.441957 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.441956    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.442245 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.442054    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d5139d6b-153a-4ba1-a0be-17cac7482f92-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.442245 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.442084    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-config-volume\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.442245 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.442172    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.442245 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.442209    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5139d6b-153a-4ba1-a0be-17cac7482f92-config-out\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.442443 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.442310    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5139d6b-153a-4ba1-a0be-17cac7482f92-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.442443 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.442345    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.442443 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.442398    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5139d6b-153a-4ba1-a0be-17cac7482f92-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.442443 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.442424    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-web-config\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.442602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.442461    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.442602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.442479    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2rpv\" (UniqueName: \"kubernetes.io/projected/d5139d6b-153a-4ba1-a0be-17cac7482f92-kube-api-access-h2rpv\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.462296 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.462276    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 04 03:17:50.543603 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.543569    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d5139d6b-153a-4ba1-a0be-17cac7482f92-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.543603 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.543605    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-config-volume\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.543839 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.543632    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.543839 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.543656    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5139d6b-153a-4ba1-a0be-17cac7482f92-config-out\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.543839 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.543696    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5139d6b-153a-4ba1-a0be-17cac7482f92-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.543839 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.543724    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.543839 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.543805    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5139d6b-153a-4ba1-a0be-17cac7482f92-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.544118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.543870    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-web-config\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.544118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.543904    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.544118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.543931    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2rpv\" (UniqueName: \"kubernetes.io/projected/d5139d6b-153a-4ba1-a0be-17cac7482f92-kube-api-access-h2rpv\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.544118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.543978    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.544118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.544019    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5139d6b-153a-4ba1-a0be-17cac7482f92-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.544118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.544051    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.544118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.544055    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d5139d6b-153a-4ba1-a0be-17cac7482f92-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.544595 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.544524    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5139d6b-153a-4ba1-a0be-17cac7482f92-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.545126 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.544771    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5139d6b-153a-4ba1-a0be-17cac7482f92-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.546568 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.546545    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5139d6b-153a-4ba1-a0be-17cac7482f92-config-out\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.546960 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.546917    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.547407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.547382    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.547494 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.547442    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-web-config\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.547642 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.547618    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.547735 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.547702    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.547735 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.547727    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5139d6b-153a-4ba1-a0be-17cac7482f92-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.547968 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.547950    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-config-volume\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.548765 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.548748    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.589052 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.589021    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-b89bc6d87-b96wk"]
Nov 04 03:17:50.592504 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.592485    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.610278 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.610247    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2rpv\" (UniqueName: \"kubernetes.io/projected/d5139d6b-153a-4ba1-a0be-17cac7482f92-kube-api-access-h2rpv\") pod \"alertmanager-main-0\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.656660 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.656624    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b89bc6d87-b96wk"]
Nov 04 03:17:50.668711 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.668688    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:17:50.689585 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.689562    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Nov 04 03:17:50.689585 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.689557    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Nov 04 03:17:50.689744 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.689593    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Nov 04 03:17:50.689744 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.689593    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Nov 04 03:17:50.689744 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.689600    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Nov 04 03:17:50.689744 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.689659    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-favgrlm1sfo91\""
Nov 04 03:17:50.689744 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.689716    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-bwccc\""
Nov 04 03:17:50.746401 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.746364    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c55446fd-0548-421c-8460-4b03026dfaa8-metrics-client-ca\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.746401 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.746402    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c55446fd-0548-421c-8460-4b03026dfaa8-tmp\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.746698 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.746428    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-grpc-tls\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.746698 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.746596    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcmgv\" (UniqueName: \"kubernetes.io/projected/c55446fd-0548-421c-8460-4b03026dfaa8-kube-api-access-pcmgv\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.746698 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.746632    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.746814 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.746721    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.746814 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.746758    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.746814 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.746785    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.746948 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.746829    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-thanos-querier-tls\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.848016 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.847980    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.848016 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.848019    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.848234 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.848042    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-thanos-querier-tls\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.848234 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.848063    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c55446fd-0548-421c-8460-4b03026dfaa8-metrics-client-ca\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.848234 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.848089    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c55446fd-0548-421c-8460-4b03026dfaa8-tmp\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.848234 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.848143    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-grpc-tls\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.848234 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.848199    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcmgv\" (UniqueName: \"kubernetes.io/projected/c55446fd-0548-421c-8460-4b03026dfaa8-kube-api-access-pcmgv\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.848234 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.848222    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.848594 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.848257    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.848594 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.848572    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c55446fd-0548-421c-8460-4b03026dfaa8-tmp\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.848962 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.848909    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c55446fd-0548-421c-8460-4b03026dfaa8-metrics-client-ca\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.850720 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.850689    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.850816 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.850727    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.851068 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.851049    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-grpc-tls\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.851200 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.851179    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-thanos-querier-tls\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.851298 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.851277    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.851439 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.851423    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c55446fd-0548-421c-8460-4b03026dfaa8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:50.910259 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:50.910228    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcmgv\" (UniqueName: \"kubernetes.io/projected/c55446fd-0548-421c-8460-4b03026dfaa8-kube-api-access-pcmgv\") pod \"thanos-querier-b89bc6d87-b96wk\" (UID: \"c55446fd-0548-421c-8460-4b03026dfaa8\") " pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:51.081873 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:51.081826    4521 generic.go:356] "Generic (PLEG): container finished" podID="b3f8b142-e183-45b1-a9b8-1581a885b0c0" containerID="27bb0e6b471981eaccab5db5c41280ae857b5c1639b9f3640618dab5b5c21f0b" exitCode=0
Nov 04 03:17:51.082055 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:51.081914    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xng65" event={"ID":"b3f8b142-e183-45b1-a9b8-1581a885b0c0","Type":"ContainerDied","Data":"27bb0e6b471981eaccab5db5c41280ae857b5c1639b9f3640618dab5b5c21f0b"}
Nov 04 03:17:51.101937 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:51.101915    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:17:51.110822 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:51.110790    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5139d6b_153a_4ba1_a0be_17cac7482f92.slice/crio-f35606e193b07c246457a06c774b7d698e189abc56f386407ef33516db5f4c65 WatchSource:0}: Error finding container f35606e193b07c246457a06c774b7d698e189abc56f386407ef33516db5f4c65: Status 404 returned error can't find the container with id f35606e193b07c246457a06c774b7d698e189abc56f386407ef33516db5f4c65
Nov 04 03:17:51.209626 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:51.209597    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:51.592400 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:51.592374    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b89bc6d87-b96wk"]
Nov 04 03:17:51.594931 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:51.594901    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc55446fd_0548_421c_8460_4b03026dfaa8.slice/crio-ae0243b85ae41fae1e56bc385ce25e6e89f7c4e5e0e91ec8ec9cdde199ce8501 WatchSource:0}: Error finding container ae0243b85ae41fae1e56bc385ce25e6e89f7c4e5e0e91ec8ec9cdde199ce8501: Status 404 returned error can't find the container with id ae0243b85ae41fae1e56bc385ce25e6e89f7c4e5e0e91ec8ec9cdde199ce8501
Nov 04 03:17:52.086970 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.086929    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5139d6b-153a-4ba1-a0be-17cac7482f92","Type":"ContainerStarted","Data":"f35606e193b07c246457a06c774b7d698e189abc56f386407ef33516db5f4c65"}
Nov 04 03:17:52.089308 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.089276    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xng65" event={"ID":"b3f8b142-e183-45b1-a9b8-1581a885b0c0","Type":"ContainerStarted","Data":"53155adc4d3675d72f6a110fb81ee945d9312efeb3555e2b76a24f17e78dea61"}
Nov 04 03:17:52.089450 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.089333    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xng65" event={"ID":"b3f8b142-e183-45b1-a9b8-1581a885b0c0","Type":"ContainerStarted","Data":"acda2fd68dd9811d9b7966bf44d5dbb4af38cd17f0bfc0593eb168792314924d"}
Nov 04 03:17:52.090548 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.090509    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk" event={"ID":"c55446fd-0548-421c-8460-4b03026dfaa8","Type":"ContainerStarted","Data":"ae0243b85ae41fae1e56bc385ce25e6e89f7c4e5e0e91ec8ec9cdde199ce8501"}
Nov 04 03:17:52.209467 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.209415    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xng65" podStartSLOduration=3.293108411 podStartE2EDuration="5.209403142s" totalImagesPullingTime="1.848380754s" totalInitContainerRuntime="67.913977ms" isStatefulPod=true podCreationTimestamp="2026-11-04 03:17:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:52.208947333 +0000 UTC m=+38.413547635" watchObservedRunningTime="2026-11-04 03:17:52.209403142 +0000 UTC m=+38.414003444"
Nov 04 03:17:52.625063 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.625025    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6849b7646f-jb6xj"]
Nov 04 03:17:52.628712 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.628691    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.681725 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.681699    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 04 03:17:52.682118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.682058    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 04 03:17:52.682118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.682075    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 04 03:17:52.682118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.682057    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-87h24\""
Nov 04 03:17:52.682118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.682107    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 04 03:17:52.682464 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.682127    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 04 03:17:52.682464 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.682143    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 04 03:17:52.682464 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.682173    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 04 03:17:52.689045 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.689023    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6849b7646f-jb6xj"]
Nov 04 03:17:52.769719 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.769682    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-console-config\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.769928 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.769752    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-service-ca\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.769928 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.769790    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/134ba14e-4203-45cb-b446-9874f583d023-tmp\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.769928 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.769816    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr5dc\" (UniqueName: \"kubernetes.io/projected/134ba14e-4203-45cb-b446-9874f583d023-kube-api-access-zr5dc\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.769928 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.769911    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/134ba14e-4203-45cb-b446-9874f583d023-console-oauth-config\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.770164 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.769946    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-oauth-serving-cert\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.770202 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.770178    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/134ba14e-4203-45cb-b446-9874f583d023-console-serving-cert\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.870759 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.870723    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-console-config\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.870957 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.870794    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-service-ca\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.870957 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.870822    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/134ba14e-4203-45cb-b446-9874f583d023-tmp\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.870957 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.870893    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr5dc\" (UniqueName: \"kubernetes.io/projected/134ba14e-4203-45cb-b446-9874f583d023-kube-api-access-zr5dc\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.870957 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.870948    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/134ba14e-4203-45cb-b446-9874f583d023-console-oauth-config\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.871188 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.870974    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-oauth-serving-cert\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.871188 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.871032    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/134ba14e-4203-45cb-b446-9874f583d023-console-serving-cert\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.871953 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.871358    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/134ba14e-4203-45cb-b446-9874f583d023-tmp\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.871953 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.871875    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-service-ca\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.871953 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.871906    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-console-config\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.872196 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.872046    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-oauth-serving-cert\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.873619 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.873595    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/134ba14e-4203-45cb-b446-9874f583d023-console-oauth-config\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.873744 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.873721    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/134ba14e-4203-45cb-b446-9874f583d023-console-serving-cert\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.875291 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.875133    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7697dbf688-n4qqr"]
Nov 04 03:17:52.879853 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.879807    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:52.933949 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.933914    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr5dc\" (UniqueName: \"kubernetes.io/projected/134ba14e-4203-45cb-b446-9874f583d023-kube-api-access-zr5dc\") pod \"console-6849b7646f-jb6xj\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") " pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.934279 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.934255    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 04 03:17:52.934279 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.934279    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-59cq7\""
Nov 04 03:17:52.934449 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.934319    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-5tvd44e9kp8ui\""
Nov 04 03:17:52.934449 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.934325    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Nov 04 03:17:52.934449 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.934365    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Nov 04 03:17:52.934449 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.934370    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Nov 04 03:17:52.940713 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.940642    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7697dbf688-n4qqr"]
Nov 04 03:17:52.941192 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.940977    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:17:52.972174 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.972137    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5lpg\" (UniqueName: \"kubernetes.io/projected/000af384-a1dc-49b1-a49c-46e1c8a3d030-kube-api-access-z5lpg\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:52.972321 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.972190    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/000af384-a1dc-49b1-a49c-46e1c8a3d030-metrics-server-audit-profiles\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:52.972321 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.972222    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/000af384-a1dc-49b1-a49c-46e1c8a3d030-secret-metrics-server-tls\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:52.972321 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.972298    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/000af384-a1dc-49b1-a49c-46e1c8a3d030-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:52.972538 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.972445    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/000af384-a1dc-49b1-a49c-46e1c8a3d030-audit-log\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:52.972538 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.972497    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/000af384-a1dc-49b1-a49c-46e1c8a3d030-secret-metrics-server-client-certs\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:52.972612 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:52.972584    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/000af384-a1dc-49b1-a49c-46e1c8a3d030-client-ca-bundle\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:53.073164 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:53.073131    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/000af384-a1dc-49b1-a49c-46e1c8a3d030-audit-log\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:53.073364 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:53.073191    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/000af384-a1dc-49b1-a49c-46e1c8a3d030-secret-metrics-server-client-certs\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:53.073414 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:53.073384    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/000af384-a1dc-49b1-a49c-46e1c8a3d030-client-ca-bundle\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:53.073506 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:53.073488    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5lpg\" (UniqueName: \"kubernetes.io/projected/000af384-a1dc-49b1-a49c-46e1c8a3d030-kube-api-access-z5lpg\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:53.073544 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:53.073525    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/000af384-a1dc-49b1-a49c-46e1c8a3d030-metrics-server-audit-profiles\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:53.073577 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:53.073558    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/000af384-a1dc-49b1-a49c-46e1c8a3d030-secret-metrics-server-tls\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:53.073613 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:53.073588    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/000af384-a1dc-49b1-a49c-46e1c8a3d030-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:53.073613 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:53.073594    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/000af384-a1dc-49b1-a49c-46e1c8a3d030-audit-log\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:53.074942 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:53.074894    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/000af384-a1dc-49b1-a49c-46e1c8a3d030-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:53.075093 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:53.075067    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/000af384-a1dc-49b1-a49c-46e1c8a3d030-metrics-server-audit-profiles\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:53.076808 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:53.076787    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/000af384-a1dc-49b1-a49c-46e1c8a3d030-client-ca-bundle\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:53.077127 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:53.077103    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/000af384-a1dc-49b1-a49c-46e1c8a3d030-secret-metrics-server-client-certs\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:53.077539 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:53.077518    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/000af384-a1dc-49b1-a49c-46e1c8a3d030-secret-metrics-server-tls\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:53.136679 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:53.136598    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5lpg\" (UniqueName: \"kubernetes.io/projected/000af384-a1dc-49b1-a49c-46e1c8a3d030-kube-api-access-z5lpg\") pod \"metrics-server-7697dbf688-n4qqr\" (UID: \"000af384-a1dc-49b1-a49c-46e1c8a3d030\") " pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:53.198102 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:53.198064    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:17:53.323359 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:53.323326    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6849b7646f-jb6xj"]
Nov 04 03:17:53.327482 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:53.327451    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod134ba14e_4203_45cb_b446_9874f583d023.slice/crio-49d47c076ee7615ebb9e6b0db0c5396dbeecdfdebf54de79241147b44b20f21c WatchSource:0}: Error finding container 49d47c076ee7615ebb9e6b0db0c5396dbeecdfdebf54de79241147b44b20f21c: Status 404 returned error can't find the container with id 49d47c076ee7615ebb9e6b0db0c5396dbeecdfdebf54de79241147b44b20f21c
Nov 04 03:17:53.581143 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:53.581106    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7697dbf688-n4qqr"]
Nov 04 03:17:53.585371 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:53.585340    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod000af384_a1dc_49b1_a49c_46e1c8a3d030.slice/crio-77615e800ec665742c838ecc2fb3296837adb420901f352e4b56d701245783eb WatchSource:0}: Error finding container 77615e800ec665742c838ecc2fb3296837adb420901f352e4b56d701245783eb: Status 404 returned error can't find the container with id 77615e800ec665742c838ecc2fb3296837adb420901f352e4b56d701245783eb
Nov 04 03:17:54.101063 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:54.101024    4521 generic.go:356] "Generic (PLEG): container finished" podID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerID="1a013fdbe83d6aaa2623f657ae0c2dccf625888540c30899a611f32437ff7411" exitCode=0
Nov 04 03:17:54.101475 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:54.101115    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5139d6b-153a-4ba1-a0be-17cac7482f92","Type":"ContainerDied","Data":"1a013fdbe83d6aaa2623f657ae0c2dccf625888540c30899a611f32437ff7411"}
Nov 04 03:17:54.102788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:54.102763    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6849b7646f-jb6xj" event={"ID":"134ba14e-4203-45cb-b446-9874f583d023","Type":"ContainerStarted","Data":"49d47c076ee7615ebb9e6b0db0c5396dbeecdfdebf54de79241147b44b20f21c"}
Nov 04 03:17:54.104038 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:54.104016    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr" event={"ID":"000af384-a1dc-49b1-a49c-46e1c8a3d030","Type":"ContainerStarted","Data":"77615e800ec665742c838ecc2fb3296837adb420901f352e4b56d701245783eb"}
Nov 04 03:17:55.111799 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:55.111759    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk" event={"ID":"c55446fd-0548-421c-8460-4b03026dfaa8","Type":"ContainerStarted","Data":"bd0f71b7b74209e11e13921113ba20cac2d5d87e4bf504e030274eabe02d9a27"}
Nov 04 03:17:55.112251 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:55.111861    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk" event={"ID":"c55446fd-0548-421c-8460-4b03026dfaa8","Type":"ContainerStarted","Data":"71414bf0b6e8d6c871decaf494d92e45919aca0ad2beb03a6119ee2a61e71031"}
Nov 04 03:17:55.112251 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:55.111939    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk" event={"ID":"c55446fd-0548-421c-8460-4b03026dfaa8","Type":"ContainerStarted","Data":"1a42f20ce10e1d303e0264c6a4eeabd40383e3b9920c9a3d43daca19762f8b4b"}
Nov 04 03:17:56.119614 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:56.119457    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr" event={"ID":"000af384-a1dc-49b1-a49c-46e1c8a3d030","Type":"ContainerStarted","Data":"d88c58b2200b71dc53449fecc6154d7062046d0452cfe623de1e1515f02545d7"}
Nov 04 03:17:56.244620 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:56.244567    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr" podStartSLOduration=1.994950331 podStartE2EDuration="4.244549927s" totalImagesPullingTime="2.249599596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:56.244507655 +0000 UTC m=+42.449107984" watchObservedRunningTime="2026-11-04 03:17:56.244549927 +0000 UTC m=+42.449150231"
Nov 04 03:17:57.125678 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:57.125635    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk" event={"ID":"c55446fd-0548-421c-8460-4b03026dfaa8","Type":"ContainerStarted","Data":"a703b35dc6cb69ac8af67981053a4f16cc58a26f4e38ab35bb3df8d3c5cdb462"}
Nov 04 03:17:57.126181 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:57.125752    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk" event={"ID":"c55446fd-0548-421c-8460-4b03026dfaa8","Type":"ContainerStarted","Data":"15250f5ca24007d19f71f123dbb90467460806bbe955fe2d69329d122005e420"}
Nov 04 03:17:57.126181 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:57.125785    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk" event={"ID":"c55446fd-0548-421c-8460-4b03026dfaa8","Type":"ContainerStarted","Data":"2df20fd6e0bc8ef456b6c931b4fd8b7a9037ead16a9e36b20500ae8fd3147726"}
Nov 04 03:17:57.126181 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:57.125972    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:17:57.130608 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:57.130576    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5139d6b-153a-4ba1-a0be-17cac7482f92","Type":"ContainerStarted","Data":"7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82"}
Nov 04 03:17:57.130747 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:57.130646    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5139d6b-153a-4ba1-a0be-17cac7482f92","Type":"ContainerStarted","Data":"66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19"}
Nov 04 03:17:57.130747 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:57.130684    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5139d6b-153a-4ba1-a0be-17cac7482f92","Type":"ContainerStarted","Data":"0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0"}
Nov 04 03:17:57.130747 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:57.130720    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5139d6b-153a-4ba1-a0be-17cac7482f92","Type":"ContainerStarted","Data":"5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0"}
Nov 04 03:17:57.130919 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:57.130761    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5139d6b-153a-4ba1-a0be-17cac7482f92","Type":"ContainerStarted","Data":"64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6"}
Nov 04 03:17:57.130919 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:57.130790    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5139d6b-153a-4ba1-a0be-17cac7482f92","Type":"ContainerStarted","Data":"2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58"}
Nov 04 03:17:57.253449 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:57.253337    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk" podStartSLOduration=2.834420777 podStartE2EDuration="7.253322342s" totalImagesPullingTime="4.418901565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:57.252502807 +0000 UTC m=+43.457103110" watchObservedRunningTime="2026-11-04 03:17:57.253322342 +0000 UTC m=+43.457922641"
Nov 04 03:17:57.383621 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:57.383555    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.655636352 podStartE2EDuration="7.383537566s" totalImagesPullingTime="4.713333323s" totalInitContainerRuntime="14.567891ms" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:57.382515846 +0000 UTC m=+43.587116212" watchObservedRunningTime="2026-11-04 03:17:57.383537566 +0000 UTC m=+43.588137868"
Nov 04 03:17:57.957620 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:57.957590    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f4dc574c4-72n8l"]
Nov 04 03:17:57.961263 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:57.961238    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.065105 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.065074    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Nov 04 03:17:58.066209 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.066177    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f4dc574c4-72n8l"]
Nov 04 03:17:58.123019 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.122973    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-config\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.123019 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.123027    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp8x6\" (UniqueName: \"kubernetes.io/projected/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-kube-api-access-hp8x6\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.123301 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.123141    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-tmp\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.123301 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.123178    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-trusted-ca-bundle\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.123301 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.123201    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-serving-cert\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.123301 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.123236    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-oauth-serving-cert\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.123301 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.123295    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-service-ca\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.123548 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.123327    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-oauth-config\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.224296 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.224207    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-config\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.224296 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.224261    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp8x6\" (UniqueName: \"kubernetes.io/projected/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-kube-api-access-hp8x6\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.224886 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.224356    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-tmp\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.224886 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.224407    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-trusted-ca-bundle\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.224886 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.224424    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-serving-cert\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.224886 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.224454    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-oauth-serving-cert\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.224886 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.224601    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-service-ca\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.224886 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.224647    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-oauth-config\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.226996 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.225237    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-config\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.226996 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.226451    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-service-ca\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.226996 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.226523    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-trusted-ca-bundle\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.226996 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.226832    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-tmp\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.227280 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.226998    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-oauth-serving-cert\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.228871 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.228831    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-oauth-config\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.229075 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.229055    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-serving-cert\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.286309 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.286276    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp8x6\" (UniqueName: \"kubernetes.io/projected/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-kube-api-access-hp8x6\") pod \"console-6f4dc574c4-72n8l\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") " pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.575615 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.575589    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:17:58.728214 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:17:58.728178    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:17:58.948896 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:58.948794    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f4dc574c4-72n8l"]
Nov 04 03:17:58.951710 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:17:58.951678    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb99e5303_fbd3_44d8_8bb0_a4db17ab80f0.slice/crio-b7cd1ce53b23872b33e94fd83e99e98f2f31bcea84084905ec1ee2136f2f4d21 WatchSource:0}: Error finding container b7cd1ce53b23872b33e94fd83e99e98f2f31bcea84084905ec1ee2136f2f4d21: Status 404 returned error can't find the container with id b7cd1ce53b23872b33e94fd83e99e98f2f31bcea84084905ec1ee2136f2f4d21
Nov 04 03:17:59.138586 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:59.138546    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f4dc574c4-72n8l" event={"ID":"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0","Type":"ContainerStarted","Data":"ddd5477a9b88c17281976e7da09c1fe1d0bb6db9b0ba804006b836d67e318df8"}
Nov 04 03:17:59.138781 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:59.138601    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f4dc574c4-72n8l" event={"ID":"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0","Type":"ContainerStarted","Data":"b7cd1ce53b23872b33e94fd83e99e98f2f31bcea84084905ec1ee2136f2f4d21"}
Nov 04 03:17:59.140119 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:59.140088    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6849b7646f-jb6xj" event={"ID":"134ba14e-4203-45cb-b446-9874f583d023","Type":"ContainerStarted","Data":"8e09c340e957c04bb58c4667f8b84aa261f2d74833cc3493b4ec57ddb2c8b77d"}
Nov 04 03:17:59.258432 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:59.258327    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6f4dc574c4-72n8l" podStartSLOduration=2.258308106 podStartE2EDuration="2.258308106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:59.257719857 +0000 UTC m=+45.462320163" watchObservedRunningTime="2026-11-04 03:17:59.258308106 +0000 UTC m=+45.462908410"
Nov 04 03:17:59.376946 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:17:59.376889    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6849b7646f-jb6xj" podStartSLOduration=2.17683277 podStartE2EDuration="7.376872941s" totalImagesPullingTime="5.200040171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:59.376389909 +0000 UTC m=+45.580990230" watchObservedRunningTime="2026-11-04 03:17:59.376872941 +0000 UTC m=+45.581473307"
Nov 04 03:18:02.942213 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:02.942174    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:18:02.942760 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:02.942239    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:18:02.947049 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:02.947027    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:18:03.140140 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:03.140111    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-b89bc6d87-b96wk"
Nov 04 03:18:03.157486 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:03.157462    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:18:03.913177 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:03.913139    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-dm5lv-xwxch"]
Nov 04 03:18:03.913509 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:03.913480    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-dm5lv-xwxch" podUID="97184dbe-af1d-408e-8dbc-0822eb2f2d8d" containerName="insights-gathering" containerID="cri-o://73e9e632fe2a67086b6eb08e1342e7886c729089d953f698fc398261e8fc080c" gracePeriod=30
Nov 04 03:18:04.163768 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:04.163697    4521 generic.go:356] "Generic (PLEG): container finished" podID="97184dbe-af1d-408e-8dbc-0822eb2f2d8d" containerID="73e9e632fe2a67086b6eb08e1342e7886c729089d953f698fc398261e8fc080c" exitCode=2
Nov 04 03:18:04.164111 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:04.163769    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-dm5lv-xwxch" event={"ID":"97184dbe-af1d-408e-8dbc-0822eb2f2d8d","Type":"ContainerDied","Data":"73e9e632fe2a67086b6eb08e1342e7886c729089d953f698fc398261e8fc080c"}
Nov 04 03:18:04.164111 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:04.163880    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-dm5lv-xwxch" event={"ID":"97184dbe-af1d-408e-8dbc-0822eb2f2d8d","Type":"ContainerDied","Data":"6e0704982824dc3fb0713a5fc64778a70fcd7b9ad8bad1b9aaf2dfcd530d119d"}
Nov 04 03:18:04.164111 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:04.163903    4521 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e0704982824dc3fb0713a5fc64778a70fcd7b9ad8bad1b9aaf2dfcd530d119d"
Nov 04 03:18:04.170110 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:04.170091    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:18:04.276425 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:04.276393    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-service-ca-bundle\") pod \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") "
Nov 04 03:18:04.276591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:04.276439    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-trusted-ca-bundle\") pod \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") "
Nov 04 03:18:04.276591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:04.276464    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-archives-path\") pod \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") "
Nov 04 03:18:04.276591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:04.276515    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-kube-api-access-bmgmq\" (UniqueName: \"kubernetes.io/projected/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-kube-api-access-bmgmq\") pod \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\" (UID: \"97184dbe-af1d-408e-8dbc-0822eb2f2d8d\") "
Nov 04 03:18:04.276863 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:04.276753    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-service-ca-bundle" pod "97184dbe-af1d-408e-8dbc-0822eb2f2d8d" (UID: "97184dbe-af1d-408e-8dbc-0822eb2f2d8d"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:18:04.276959 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:04.276866    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-archives-path" pod "97184dbe-af1d-408e-8dbc-0822eb2f2d8d" (UID: "97184dbe-af1d-408e-8dbc-0822eb2f2d8d"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:18:04.276959 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:04.276914    4521 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-service-ca-bundle\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:04.277072 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:04.276996    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-trusted-ca-bundle" pod "97184dbe-af1d-408e-8dbc-0822eb2f2d8d" (UID: "97184dbe-af1d-408e-8dbc-0822eb2f2d8d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:18:04.278878 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:04.278853    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-kube-api-access-bmgmq" pod "97184dbe-af1d-408e-8dbc-0822eb2f2d8d" (UID: "97184dbe-af1d-408e-8dbc-0822eb2f2d8d"). InnerVolumeSpecName "kube-api-access-bmgmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:18:04.377453 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:04.377423    4521 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-trusted-ca-bundle\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:04.377453 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:04.377448    4521 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-archives-path\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:04.377453 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:04.377457    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmgmq\" (UniqueName: \"kubernetes.io/projected/97184dbe-af1d-408e-8dbc-0822eb2f2d8d-kube-api-access-bmgmq\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:05.166693 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:05.166660    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-dm5lv-xwxch"
Nov 04 03:18:05.329924 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:05.329893    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-dm5lv-xwxch"]
Nov 04 03:18:05.380509 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:05.380477    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-dm5lv-xwxch"]
Nov 04 03:18:05.909386 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:05.909353    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97184dbe-af1d-408e-8dbc-0822eb2f2d8d" path="/var/lib/kubelet/pods/97184dbe-af1d-408e-8dbc-0822eb2f2d8d/volumes"
Nov 04 03:18:08.576489 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:08.576446    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:18:08.576896 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:08.576515    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:18:08.581000 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:08.580978    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:18:09.187342 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:09.187312    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:18:09.357627 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:09.357591    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6849b7646f-jb6xj"]
Nov 04 03:18:13.198358 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:13.198324    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:18:13.198837 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:13.198378    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:18:28.729330 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:18:28.729291    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:18:33.203710 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:33.203678    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:18:33.207251 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:33.207232    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7697dbf688-n4qqr"
Nov 04 03:18:34.378763 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.378719    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6849b7646f-jb6xj" podUID="134ba14e-4203-45cb-b446-9874f583d023" containerName="console" containerID="cri-o://8e09c340e957c04bb58c4667f8b84aa261f2d74833cc3493b4ec57ddb2c8b77d" gracePeriod=15
Nov 04 03:18:34.634994 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.634934    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:18:34.714361 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.714332    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-oauth-serving-cert\") pod \"134ba14e-4203-45cb-b446-9874f583d023\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") "
Nov 04 03:18:34.714522 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.714380    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-console-config\" (UniqueName: \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-console-config\") pod \"134ba14e-4203-45cb-b446-9874f583d023\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") "
Nov 04 03:18:34.714522 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.714414    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/134ba14e-4203-45cb-b446-9874f583d023-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/134ba14e-4203-45cb-b446-9874f583d023-console-serving-cert\") pod \"134ba14e-4203-45cb-b446-9874f583d023\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") "
Nov 04 03:18:34.714522 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.714464    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/134ba14e-4203-45cb-b446-9874f583d023-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/134ba14e-4203-45cb-b446-9874f583d023-console-oauth-config\") pod \"134ba14e-4203-45cb-b446-9874f583d023\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") "
Nov 04 03:18:34.714522 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.714513    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/134ba14e-4203-45cb-b446-9874f583d023-kube-api-access-zr5dc\" (UniqueName: \"kubernetes.io/projected/134ba14e-4203-45cb-b446-9874f583d023-kube-api-access-zr5dc\") pod \"134ba14e-4203-45cb-b446-9874f583d023\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") "
Nov 04 03:18:34.714742 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.714558    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/134ba14e-4203-45cb-b446-9874f583d023-tmp\" (UniqueName: \"kubernetes.io/empty-dir/134ba14e-4203-45cb-b446-9874f583d023-tmp\") pod \"134ba14e-4203-45cb-b446-9874f583d023\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") "
Nov 04 03:18:34.714742 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.714618    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-service-ca\" (UniqueName: \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-service-ca\") pod \"134ba14e-4203-45cb-b446-9874f583d023\" (UID: \"134ba14e-4203-45cb-b446-9874f583d023\") "
Nov 04 03:18:34.714838 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.714781    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-oauth-serving-cert" pod "134ba14e-4203-45cb-b446-9874f583d023" (UID: "134ba14e-4203-45cb-b446-9874f583d023"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:18:34.714986 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.714905    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/134ba14e-4203-45cb-b446-9874f583d023-tmp" pod "134ba14e-4203-45cb-b446-9874f583d023" (UID: "134ba14e-4203-45cb-b446-9874f583d023"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:18:34.714986 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.714932    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-console-config" pod "134ba14e-4203-45cb-b446-9874f583d023" (UID: "134ba14e-4203-45cb-b446-9874f583d023"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:18:34.715155 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.715132    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-service-ca" pod "134ba14e-4203-45cb-b446-9874f583d023" (UID: "134ba14e-4203-45cb-b446-9874f583d023"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:18:34.716755 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.716724    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/134ba14e-4203-45cb-b446-9874f583d023-console-serving-cert" pod "134ba14e-4203-45cb-b446-9874f583d023" (UID: "134ba14e-4203-45cb-b446-9874f583d023"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:18:34.716867 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.716738    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/134ba14e-4203-45cb-b446-9874f583d023-console-oauth-config" pod "134ba14e-4203-45cb-b446-9874f583d023" (UID: "134ba14e-4203-45cb-b446-9874f583d023"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:18:34.716867 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.716825    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/134ba14e-4203-45cb-b446-9874f583d023-kube-api-access-zr5dc" pod "134ba14e-4203-45cb-b446-9874f583d023" (UID: "134ba14e-4203-45cb-b446-9874f583d023"). InnerVolumeSpecName "kube-api-access-zr5dc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:18:34.815920 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.815888    4521 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/134ba14e-4203-45cb-b446-9874f583d023-console-oauth-config\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:34.815920 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.815914    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zr5dc\" (UniqueName: \"kubernetes.io/projected/134ba14e-4203-45cb-b446-9874f583d023-kube-api-access-zr5dc\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:34.815920 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.815926    4521 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/134ba14e-4203-45cb-b446-9874f583d023-tmp\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:34.816145 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.815938    4521 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-service-ca\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:34.816145 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.815946    4521 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-oauth-serving-cert\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:34.816145 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.815955    4521 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/134ba14e-4203-45cb-b446-9874f583d023-console-config\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:34.816145 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:34.815964    4521 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/134ba14e-4203-45cb-b446-9874f583d023-console-serving-cert\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:35.276511 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:35.276476    4521 generic.go:356] "Generic (PLEG): container finished" podID="134ba14e-4203-45cb-b446-9874f583d023" containerID="8e09c340e957c04bb58c4667f8b84aa261f2d74833cc3493b4ec57ddb2c8b77d" exitCode=0
Nov 04 03:18:35.276706 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:35.276529    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6849b7646f-jb6xj" event={"ID":"134ba14e-4203-45cb-b446-9874f583d023","Type":"ContainerDied","Data":"8e09c340e957c04bb58c4667f8b84aa261f2d74833cc3493b4ec57ddb2c8b77d"}
Nov 04 03:18:35.276706 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:35.276533    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6849b7646f-jb6xj"
Nov 04 03:18:35.276706 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:35.276572    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6849b7646f-jb6xj" event={"ID":"134ba14e-4203-45cb-b446-9874f583d023","Type":"ContainerDied","Data":"49d47c076ee7615ebb9e6b0db0c5396dbeecdfdebf54de79241147b44b20f21c"}
Nov 04 03:18:35.276706 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:35.276590    4521 scope.go:122] "RemoveContainer" containerID="8e09c340e957c04bb58c4667f8b84aa261f2d74833cc3493b4ec57ddb2c8b77d"
Nov 04 03:18:35.286632 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:35.286613    4521 scope.go:122] "RemoveContainer" containerID="8e09c340e957c04bb58c4667f8b84aa261f2d74833cc3493b4ec57ddb2c8b77d"
Nov 04 03:18:35.286917 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:18:35.286896    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e09c340e957c04bb58c4667f8b84aa261f2d74833cc3493b4ec57ddb2c8b77d\": container with ID starting with 8e09c340e957c04bb58c4667f8b84aa261f2d74833cc3493b4ec57ddb2c8b77d not found: ID does not exist" containerID="8e09c340e957c04bb58c4667f8b84aa261f2d74833cc3493b4ec57ddb2c8b77d"
Nov 04 03:18:35.286988 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:35.286927    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e09c340e957c04bb58c4667f8b84aa261f2d74833cc3493b4ec57ddb2c8b77d"} err="failed to get container status \"8e09c340e957c04bb58c4667f8b84aa261f2d74833cc3493b4ec57ddb2c8b77d\": rpc error: code = NotFound desc = could not find container \"8e09c340e957c04bb58c4667f8b84aa261f2d74833cc3493b4ec57ddb2c8b77d\": container with ID starting with 8e09c340e957c04bb58c4667f8b84aa261f2d74833cc3493b4ec57ddb2c8b77d not found: ID does not exist"
Nov 04 03:18:35.441640 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:35.441601    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6849b7646f-jb6xj"]
Nov 04 03:18:35.492244 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:35.492210    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6849b7646f-jb6xj"]
Nov 04 03:18:35.913180 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:18:35.913143    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="134ba14e-4203-45cb-b446-9874f583d023" path="/var/lib/kubelet/pods/134ba14e-4203-45cb-b446-9874f583d023/volumes"
Nov 04 03:18:58.728648 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:18:58.728612    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:19:04.130969 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:04.130928    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:19:04.131601 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:04.131357    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="alertmanager" containerID="cri-o://2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58" gracePeriod=120
Nov 04 03:19:04.131601 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:04.131404    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="kube-rbac-proxy-metric" containerID="cri-o://66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19" gracePeriod=120
Nov 04 03:19:04.131601 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:04.131432    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="kube-rbac-proxy" containerID="cri-o://0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0" gracePeriod=120
Nov 04 03:19:04.131601 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:04.131472    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="config-reloader" containerID="cri-o://64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6" gracePeriod=120
Nov 04 03:19:04.131601 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:04.131489    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="prom-label-proxy" containerID="cri-o://7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82" gracePeriod=120
Nov 04 03:19:04.131601 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:04.131411    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="kube-rbac-proxy-web" containerID="cri-o://5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0" gracePeriod=120
Nov 04 03:19:04.381460 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:04.381376    4521 generic.go:356] "Generic (PLEG): container finished" podID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerID="7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82" exitCode=0
Nov 04 03:19:04.381460 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:04.381404    4521 generic.go:356] "Generic (PLEG): container finished" podID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerID="0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0" exitCode=0
Nov 04 03:19:04.381460 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:04.381410    4521 generic.go:356] "Generic (PLEG): container finished" podID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerID="64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6" exitCode=0
Nov 04 03:19:04.381460 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:04.381416    4521 generic.go:356] "Generic (PLEG): container finished" podID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerID="2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58" exitCode=0
Nov 04 03:19:04.381460 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:04.381445    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5139d6b-153a-4ba1-a0be-17cac7482f92","Type":"ContainerDied","Data":"7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82"}
Nov 04 03:19:04.381718 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:04.381509    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5139d6b-153a-4ba1-a0be-17cac7482f92","Type":"ContainerDied","Data":"0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0"}
Nov 04 03:19:04.381718 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:04.381536    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5139d6b-153a-4ba1-a0be-17cac7482f92","Type":"ContainerDied","Data":"64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6"}
Nov 04 03:19:04.381718 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:04.381561    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5139d6b-153a-4ba1-a0be-17cac7482f92","Type":"ContainerDied","Data":"2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58"}
Nov 04 03:19:05.381446 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.381420    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:05.387935 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.387898    4521 generic.go:356] "Generic (PLEG): container finished" podID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerID="66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19" exitCode=0
Nov 04 03:19:05.387935 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.387929    4521 generic.go:356] "Generic (PLEG): container finished" podID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerID="5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0" exitCode=0
Nov 04 03:19:05.388085 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.387973    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:05.388085 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.387970    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5139d6b-153a-4ba1-a0be-17cac7482f92","Type":"ContainerDied","Data":"66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19"}
Nov 04 03:19:05.388165 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.388096    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5139d6b-153a-4ba1-a0be-17cac7482f92","Type":"ContainerDied","Data":"5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0"}
Nov 04 03:19:05.388165 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.388127    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5139d6b-153a-4ba1-a0be-17cac7482f92","Type":"ContainerDied","Data":"f35606e193b07c246457a06c774b7d698e189abc56f386407ef33516db5f4c65"}
Nov 04 03:19:05.388165 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.388159    4521 scope.go:122] "RemoveContainer" containerID="7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82"
Nov 04 03:19:05.396862 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.396810    4521 scope.go:122] "RemoveContainer" containerID="66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19"
Nov 04 03:19:05.403936 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.403907    4521 scope.go:122] "RemoveContainer" containerID="0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0"
Nov 04 03:19:05.410270 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.410254    4521 scope.go:122] "RemoveContainer" containerID="5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0"
Nov 04 03:19:05.416274 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.416257    4521 scope.go:122] "RemoveContainer" containerID="64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6"
Nov 04 03:19:05.422799 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.422783    4521 scope.go:122] "RemoveContainer" containerID="2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58"
Nov 04 03:19:05.429474 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.429458    4521 scope.go:122] "RemoveContainer" containerID="1a013fdbe83d6aaa2623f657ae0c2dccf625888540c30899a611f32437ff7411"
Nov 04 03:19:05.436191 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.436176    4521 scope.go:122] "RemoveContainer" containerID="7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82"
Nov 04 03:19:05.436447 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:19:05.436429    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82\": container with ID starting with 7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82 not found: ID does not exist" containerID="7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82"
Nov 04 03:19:05.436513 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.436461    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82"} err="failed to get container status \"7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82\": rpc error: code = NotFound desc = could not find container \"7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82\": container with ID starting with 7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82 not found: ID does not exist"
Nov 04 03:19:05.436513 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.436489    4521 scope.go:122] "RemoveContainer" containerID="66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19"
Nov 04 03:19:05.436747 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:19:05.436730    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19\": container with ID starting with 66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19 not found: ID does not exist" containerID="66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19"
Nov 04 03:19:05.436819 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.436756    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19"} err="failed to get container status \"66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19\": rpc error: code = NotFound desc = could not find container \"66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19\": container with ID starting with 66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19 not found: ID does not exist"
Nov 04 03:19:05.436819 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.436777    4521 scope.go:122] "RemoveContainer" containerID="0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0"
Nov 04 03:19:05.437020 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:19:05.437005    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0\": container with ID starting with 0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0 not found: ID does not exist" containerID="0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0"
Nov 04 03:19:05.437058 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.437025    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0"} err="failed to get container status \"0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0\": rpc error: code = NotFound desc = could not find container \"0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0\": container with ID starting with 0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0 not found: ID does not exist"
Nov 04 03:19:05.437058 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.437046    4521 scope.go:122] "RemoveContainer" containerID="5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0"
Nov 04 03:19:05.437245 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:19:05.437228    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0\": container with ID starting with 5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0 not found: ID does not exist" containerID="5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0"
Nov 04 03:19:05.437311 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.437252    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0"} err="failed to get container status \"5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0\": rpc error: code = NotFound desc = could not find container \"5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0\": container with ID starting with 5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0 not found: ID does not exist"
Nov 04 03:19:05.437311 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.437270    4521 scope.go:122] "RemoveContainer" containerID="64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6"
Nov 04 03:19:05.437506 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:19:05.437491    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6\": container with ID starting with 64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6 not found: ID does not exist" containerID="64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6"
Nov 04 03:19:05.437552 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.437510    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6"} err="failed to get container status \"64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6\": rpc error: code = NotFound desc = could not find container \"64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6\": container with ID starting with 64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6 not found: ID does not exist"
Nov 04 03:19:05.437552 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.437522    4521 scope.go:122] "RemoveContainer" containerID="2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58"
Nov 04 03:19:05.437691 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:19:05.437678    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58\": container with ID starting with 2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58 not found: ID does not exist" containerID="2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58"
Nov 04 03:19:05.437726 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.437692    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58"} err="failed to get container status \"2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58\": rpc error: code = NotFound desc = could not find container \"2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58\": container with ID starting with 2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58 not found: ID does not exist"
Nov 04 03:19:05.437726 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.437703    4521 scope.go:122] "RemoveContainer" containerID="1a013fdbe83d6aaa2623f657ae0c2dccf625888540c30899a611f32437ff7411"
Nov 04 03:19:05.437918 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:19:05.437904    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a013fdbe83d6aaa2623f657ae0c2dccf625888540c30899a611f32437ff7411\": container with ID starting with 1a013fdbe83d6aaa2623f657ae0c2dccf625888540c30899a611f32437ff7411 not found: ID does not exist" containerID="1a013fdbe83d6aaa2623f657ae0c2dccf625888540c30899a611f32437ff7411"
Nov 04 03:19:05.437969 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.437920    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a013fdbe83d6aaa2623f657ae0c2dccf625888540c30899a611f32437ff7411"} err="failed to get container status \"1a013fdbe83d6aaa2623f657ae0c2dccf625888540c30899a611f32437ff7411\": rpc error: code = NotFound desc = could not find container \"1a013fdbe83d6aaa2623f657ae0c2dccf625888540c30899a611f32437ff7411\": container with ID starting with 1a013fdbe83d6aaa2623f657ae0c2dccf625888540c30899a611f32437ff7411 not found: ID does not exist"
Nov 04 03:19:05.437969 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.437931    4521 scope.go:122] "RemoveContainer" containerID="7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82"
Nov 04 03:19:05.438103 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.438088    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82"} err="failed to get container status \"7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82\": rpc error: code = NotFound desc = could not find container \"7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82\": container with ID starting with 7a6bc391095ac65b0a5b9212d78f52dfefd47df86463fbc56a1916d03d8cad82 not found: ID does not exist"
Nov 04 03:19:05.438150 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.438102    4521 scope.go:122] "RemoveContainer" containerID="66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19"
Nov 04 03:19:05.438322 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.438296    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19"} err="failed to get container status \"66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19\": rpc error: code = NotFound desc = could not find container \"66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19\": container with ID starting with 66ad032857c1e32fe1616213b8d884fbea63a93b406c6f1adb6b8ff0eef25c19 not found: ID does not exist"
Nov 04 03:19:05.438365 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.438322    4521 scope.go:122] "RemoveContainer" containerID="0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0"
Nov 04 03:19:05.438531 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.438510    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0"} err="failed to get container status \"0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0\": rpc error: code = NotFound desc = could not find container \"0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0\": container with ID starting with 0477abd9ff4e1dd6f65dd37d28b9ea727d52472025a5943589398ef936200ef0 not found: ID does not exist"
Nov 04 03:19:05.438614 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.438532    4521 scope.go:122] "RemoveContainer" containerID="5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0"
Nov 04 03:19:05.438748 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.438729    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0"} err="failed to get container status \"5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0\": rpc error: code = NotFound desc = could not find container \"5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0\": container with ID starting with 5c54133b71f3e1891e7117f6349dff178ae9e04d90bcc9fbeb2ae572551074a0 not found: ID does not exist"
Nov 04 03:19:05.438814 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.438750    4521 scope.go:122] "RemoveContainer" containerID="64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6"
Nov 04 03:19:05.438999 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.438983    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6"} err="failed to get container status \"64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6\": rpc error: code = NotFound desc = could not find container \"64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6\": container with ID starting with 64977b7dd342ac9a82a9bd597d091f9c9776e39fe0588e71e966c205fc8bd8f6 not found: ID does not exist"
Nov 04 03:19:05.439044 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.439000    4521 scope.go:122] "RemoveContainer" containerID="2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58"
Nov 04 03:19:05.439227 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.439205    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58"} err="failed to get container status \"2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58\": rpc error: code = NotFound desc = could not find container \"2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58\": container with ID starting with 2e6da2e5fce2507207fffb05cab94a2663b6ff2e0f6ed160b28b7e6d6d55cd58 not found: ID does not exist"
Nov 04 03:19:05.439275 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.439228    4521 scope.go:122] "RemoveContainer" containerID="1a013fdbe83d6aaa2623f657ae0c2dccf625888540c30899a611f32437ff7411"
Nov 04 03:19:05.439438 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.439418    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a013fdbe83d6aaa2623f657ae0c2dccf625888540c30899a611f32437ff7411"} err="failed to get container status \"1a013fdbe83d6aaa2623f657ae0c2dccf625888540c30899a611f32437ff7411\": rpc error: code = NotFound desc = could not find container \"1a013fdbe83d6aaa2623f657ae0c2dccf625888540c30899a611f32437ff7411\": container with ID starting with 1a013fdbe83d6aaa2623f657ae0c2dccf625888540c30899a611f32437ff7411 not found: ID does not exist"
Nov 04 03:19:05.569064 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.569032    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-main-tls\") pod \"d5139d6b-153a-4ba1-a0be-17cac7482f92\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") "
Nov 04 03:19:05.569237 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.569074    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5139d6b-153a-4ba1-a0be-17cac7482f92-tls-assets\" (UniqueName: \"kubernetes.io/projected/d5139d6b-153a-4ba1-a0be-17cac7482f92-tls-assets\") pod \"d5139d6b-153a-4ba1-a0be-17cac7482f92\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") "
Nov 04 03:19:05.569237 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.569099    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5139d6b-153a-4ba1-a0be-17cac7482f92-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5139d6b-153a-4ba1-a0be-17cac7482f92-metrics-client-ca\") pod \"d5139d6b-153a-4ba1-a0be-17cac7482f92\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") "
Nov 04 03:19:05.569237 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.569165    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d5139d6b-153a-4ba1-a0be-17cac7482f92\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") "
Nov 04 03:19:05.569237 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.569195    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-config-volume\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-config-volume\") pod \"d5139d6b-153a-4ba1-a0be-17cac7482f92\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") "
Nov 04 03:19:05.569438 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.569250    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5139d6b-153a-4ba1-a0be-17cac7482f92-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d5139d6b-153a-4ba1-a0be-17cac7482f92-alertmanager-main-db\") pod \"d5139d6b-153a-4ba1-a0be-17cac7482f92\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") "
Nov 04 03:19:05.569438 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.569283    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-cluster-tls-config\") pod \"d5139d6b-153a-4ba1-a0be-17cac7482f92\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") "
Nov 04 03:19:05.569438 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.569332    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5139d6b-153a-4ba1-a0be-17cac7482f92-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5139d6b-153a-4ba1-a0be-17cac7482f92-alertmanager-trusted-ca-bundle\") pod \"d5139d6b-153a-4ba1-a0be-17cac7482f92\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") "
Nov 04 03:19:05.569598 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.569553    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5139d6b-153a-4ba1-a0be-17cac7482f92-metrics-client-ca" pod "d5139d6b-153a-4ba1-a0be-17cac7482f92" (UID: "d5139d6b-153a-4ba1-a0be-17cac7482f92"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:19:05.569647 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.569599    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5139d6b-153a-4ba1-a0be-17cac7482f92-alertmanager-main-db" pod "d5139d6b-153a-4ba1-a0be-17cac7482f92" (UID: "d5139d6b-153a-4ba1-a0be-17cac7482f92"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:19:05.569647 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.569624    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-web-config\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-web-config\") pod \"d5139d6b-153a-4ba1-a0be-17cac7482f92\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") "
Nov 04 03:19:05.569753 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.569657    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy-web\") pod \"d5139d6b-153a-4ba1-a0be-17cac7482f92\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") "
Nov 04 03:19:05.569753 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.569709    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5139d6b-153a-4ba1-a0be-17cac7482f92-kube-api-access-h2rpv\" (UniqueName: \"kubernetes.io/projected/d5139d6b-153a-4ba1-a0be-17cac7482f92-kube-api-access-h2rpv\") pod \"d5139d6b-153a-4ba1-a0be-17cac7482f92\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") "
Nov 04 03:19:05.569885 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.569751    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5139d6b-153a-4ba1-a0be-17cac7482f92-config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5139d6b-153a-4ba1-a0be-17cac7482f92-config-out\") pod \"d5139d6b-153a-4ba1-a0be-17cac7482f92\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") "
Nov 04 03:19:05.569885 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.569787    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy\") pod \"d5139d6b-153a-4ba1-a0be-17cac7482f92\" (UID: \"d5139d6b-153a-4ba1-a0be-17cac7482f92\") "
Nov 04 03:19:05.570017 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.569989    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5139d6b-153a-4ba1-a0be-17cac7482f92-alertmanager-trusted-ca-bundle" pod "d5139d6b-153a-4ba1-a0be-17cac7482f92" (UID: "d5139d6b-153a-4ba1-a0be-17cac7482f92"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:19:05.570099 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.570036    4521 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d5139d6b-153a-4ba1-a0be-17cac7482f92-alertmanager-main-db\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:05.570099 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.570059    4521 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5139d6b-153a-4ba1-a0be-17cac7482f92-metrics-client-ca\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:05.573106 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.573022    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy" pod "d5139d6b-153a-4ba1-a0be-17cac7482f92" (UID: "d5139d6b-153a-4ba1-a0be-17cac7482f92"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:05.573106 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.573034    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5139d6b-153a-4ba1-a0be-17cac7482f92-config-out" pod "d5139d6b-153a-4ba1-a0be-17cac7482f92" (UID: "d5139d6b-153a-4ba1-a0be-17cac7482f92"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:19:05.573106 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.573067    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-main-tls" pod "d5139d6b-153a-4ba1-a0be-17cac7482f92" (UID: "d5139d6b-153a-4ba1-a0be-17cac7482f92"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:05.573106 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.573082    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-config-volume" pod "d5139d6b-153a-4ba1-a0be-17cac7482f92" (UID: "d5139d6b-153a-4ba1-a0be-17cac7482f92"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:05.573311 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.573108    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5139d6b-153a-4ba1-a0be-17cac7482f92-tls-assets" pod "d5139d6b-153a-4ba1-a0be-17cac7482f92" (UID: "d5139d6b-153a-4ba1-a0be-17cac7482f92"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:19:05.573311 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.573120    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5139d6b-153a-4ba1-a0be-17cac7482f92-kube-api-access-h2rpv" pod "d5139d6b-153a-4ba1-a0be-17cac7482f92" (UID: "d5139d6b-153a-4ba1-a0be-17cac7482f92"). InnerVolumeSpecName "kube-api-access-h2rpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:19:05.573311 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.573203    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy-metric" pod "d5139d6b-153a-4ba1-a0be-17cac7482f92" (UID: "d5139d6b-153a-4ba1-a0be-17cac7482f92"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:05.573655 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.573634    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy-web" pod "d5139d6b-153a-4ba1-a0be-17cac7482f92" (UID: "d5139d6b-153a-4ba1-a0be-17cac7482f92"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:05.576645 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.576624    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-cluster-tls-config" pod "d5139d6b-153a-4ba1-a0be-17cac7482f92" (UID: "d5139d6b-153a-4ba1-a0be-17cac7482f92"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:05.582866 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.582832    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-web-config" pod "d5139d6b-153a-4ba1-a0be-17cac7482f92" (UID: "d5139d6b-153a-4ba1-a0be-17cac7482f92"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:05.671355 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.671327    4521 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-main-tls\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:05.671355 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.671353    4521 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5139d6b-153a-4ba1-a0be-17cac7482f92-tls-assets\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:05.671544 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.671364    4521 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:05.671544 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.671383    4521 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-config-volume\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:05.671544 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.671393    4521 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-cluster-tls-config\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:05.671544 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.671403    4521 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5139d6b-153a-4ba1-a0be-17cac7482f92-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:05.671544 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.671411    4521 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-web-config\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:05.671544 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.671419    4521 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:05.671544 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.671429    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2rpv\" (UniqueName: \"kubernetes.io/projected/d5139d6b-153a-4ba1-a0be-17cac7482f92-kube-api-access-h2rpv\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:05.671544 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.671437    4521 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5139d6b-153a-4ba1-a0be-17cac7482f92-config-out\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:05.671544 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.671445    4521 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5139d6b-153a-4ba1-a0be-17cac7482f92-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:05.810614 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.810569    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:19:05.814980 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.814956    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:19:05.844919 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.844839    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:19:05.845127 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845114    4521 cpu_manager.go:414] "containerMap: removing container" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="kube-rbac-proxy-metric"
Nov 04 03:19:05.845175 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845128    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="kube-rbac-proxy-metric"
Nov 04 03:19:05.845175 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845141    4521 cpu_manager.go:414] "containerMap: removing container" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="config-reloader"
Nov 04 03:19:05.845175 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845147    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="config-reloader"
Nov 04 03:19:05.845175 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845153    4521 cpu_manager.go:414] "containerMap: removing container" podUID="97184dbe-af1d-408e-8dbc-0822eb2f2d8d" containerName="insights-gathering"
Nov 04 03:19:05.845175 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845160    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97184dbe-af1d-408e-8dbc-0822eb2f2d8d" containerName="insights-gathering"
Nov 04 03:19:05.845175 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845172    4521 cpu_manager.go:414] "containerMap: removing container" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="alertmanager"
Nov 04 03:19:05.845175 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845177    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="alertmanager"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845186    4521 cpu_manager.go:414] "containerMap: removing container" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="kube-rbac-proxy"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845191    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="kube-rbac-proxy"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845199    4521 cpu_manager.go:414] "containerMap: removing container" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="init-config-reloader"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845204    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="init-config-reloader"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845212    4521 cpu_manager.go:414] "containerMap: removing container" podUID="134ba14e-4203-45cb-b446-9874f583d023" containerName="console"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845216    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="134ba14e-4203-45cb-b446-9874f583d023" containerName="console"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845225    4521 cpu_manager.go:414] "containerMap: removing container" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="prom-label-proxy"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845230    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="prom-label-proxy"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845238    4521 cpu_manager.go:414] "containerMap: removing container" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="kube-rbac-proxy-web"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845242    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="kube-rbac-proxy-web"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845284    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="alertmanager"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845293    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="134ba14e-4203-45cb-b446-9874f583d023" containerName="console"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845299    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="prom-label-proxy"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845304    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="kube-rbac-proxy-web"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845311    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="kube-rbac-proxy-metric"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845316    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="config-reloader"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845322    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="97184dbe-af1d-408e-8dbc-0822eb2f2d8d" containerName="insights-gathering"
Nov 04 03:19:05.845393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.845328    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" containerName="kube-rbac-proxy"
Nov 04 03:19:05.851025 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.850995    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:05.904969 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.904894    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 04 03:19:05.904969 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.904911    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 04 03:19:05.904969 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.904931    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 04 03:19:05.905216 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.905009    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-c24br\""
Nov 04 03:19:05.905216 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.905026    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 04 03:19:05.905216 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.905105    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 04 03:19:05.905779 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.905756    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 04 03:19:05.905917 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.905788    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 04 03:19:05.905917 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.905801    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 04 03:19:05.906247 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.906232    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 04 03:19:05.909293 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.909272    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5139d6b-153a-4ba1-a0be-17cac7482f92" path="/var/lib/kubelet/pods/d5139d6b-153a-4ba1-a0be-17cac7482f92/volumes"
Nov 04 03:19:05.956238 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.956208    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:19:05.974008 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.973977    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:05.974165 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.974020    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s92x8\" (UniqueName: \"kubernetes.io/projected/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-kube-api-access-s92x8\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:05.974165 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.974095    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:05.974165 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.974134    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:05.974165 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.974153    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:05.974300 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.974175    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-config-volume\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:05.974300 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.974201    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:05.974300 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.974277    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:05.974390 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.974327    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-web-config\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:05.974390 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.974346    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:05.974390 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.974370    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-config-out\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:05.974390 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.974388    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:05.974507 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:05.974435    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.075322 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.075269    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-web-config\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.075322 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.075321    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.075584 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.075348    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-config-out\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.075584 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.075375    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.075584 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.075392    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.075584 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.075432    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.075584 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.075466    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s92x8\" (UniqueName: \"kubernetes.io/projected/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-kube-api-access-s92x8\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.075584 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.075516    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.075584 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.075545    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.075584 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.075566    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.076764 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.075622    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-config-volume\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.076764 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.075657    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.076764 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.075684    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.076764 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.076388    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.076994 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.076887    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.076994 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.076891    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.078828 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.078803    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.078948 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.078890    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.078948 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.078896    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.079075 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.078999    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-web-config\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.079135 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.079093    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.079401 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.079375    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.079681 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.079663    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-config-out\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.079749 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.079705    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.080430 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.080404    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-config-volume\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.138662 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.138587    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s92x8\" (UniqueName: \"kubernetes.io/projected/fcfc254d-52a4-4beb-8772-d2ba0ba39cd1-kube-api-access-s92x8\") pod \"alertmanager-main-0\" (UID: \"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.172016 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.171986    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:19:06.602412 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:06.602374    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:19:06.605525 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:19:06.605505    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcfc254d_52a4_4beb_8772_d2ba0ba39cd1.slice/crio-3d54b82f22e7dda04d13c4199e10e590f07a57eeb5e2d036a24d874319815404 WatchSource:0}: Error finding container 3d54b82f22e7dda04d13c4199e10e590f07a57eeb5e2d036a24d874319815404: Status 404 returned error can't find the container with id 3d54b82f22e7dda04d13c4199e10e590f07a57eeb5e2d036a24d874319815404
Nov 04 03:19:07.397404 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.397368    4521 generic.go:356] "Generic (PLEG): container finished" podID="fcfc254d-52a4-4beb-8772-d2ba0ba39cd1" containerID="6e126c6361650f5799391eaf235dcc607b3eed47070308cebb5635ee0f00ed3e" exitCode=0
Nov 04 03:19:07.397548 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.397455    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1","Type":"ContainerDied","Data":"6e126c6361650f5799391eaf235dcc607b3eed47070308cebb5635ee0f00ed3e"}
Nov 04 03:19:07.397548 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.397529    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1","Type":"ContainerStarted","Data":"3d54b82f22e7dda04d13c4199e10e590f07a57eeb5e2d036a24d874319815404"}
Nov 04 03:19:07.712372 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.712348    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"]
Nov 04 03:19:07.716098 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.716076    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.769163 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.769139    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Nov 04 03:19:07.769163 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.769139    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Nov 04 03:19:07.769314 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.769293    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-q7glb\""
Nov 04 03:19:07.769358 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.769316    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Nov 04 03:19:07.769358 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.769340    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Nov 04 03:19:07.769430 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.769385    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Nov 04 03:19:07.820545 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.820510    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Nov 04 03:19:07.822030 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.821999    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"]
Nov 04 03:19:07.889420 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.889382    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c4dce54-5199-40ce-abfd-dae5daab91ef-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.889627 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.889436    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4dce54-5199-40ce-abfd-dae5daab91ef-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.889627 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.889522    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c4dce54-5199-40ce-abfd-dae5daab91ef-telemeter-client-tls\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.889744 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.889628    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvrcq\" (UniqueName: \"kubernetes.io/projected/9c4dce54-5199-40ce-abfd-dae5daab91ef-kube-api-access-kvrcq\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.889744 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.889672    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c4dce54-5199-40ce-abfd-dae5daab91ef-metrics-client-ca\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.889744 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.889729    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c4dce54-5199-40ce-abfd-dae5daab91ef-federate-client-tls\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.889900 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.889773    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4dce54-5199-40ce-abfd-dae5daab91ef-serving-certs-ca-bundle\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.889900 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.889802    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c4dce54-5199-40ce-abfd-dae5daab91ef-secret-telemeter-client\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.990629 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.990541    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c4dce54-5199-40ce-abfd-dae5daab91ef-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.990629 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.990583    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4dce54-5199-40ce-abfd-dae5daab91ef-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.990629 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.990622    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c4dce54-5199-40ce-abfd-dae5daab91ef-telemeter-client-tls\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.990941 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.990651    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvrcq\" (UniqueName: \"kubernetes.io/projected/9c4dce54-5199-40ce-abfd-dae5daab91ef-kube-api-access-kvrcq\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.990941 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.990682    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c4dce54-5199-40ce-abfd-dae5daab91ef-metrics-client-ca\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.990941 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.990724    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c4dce54-5199-40ce-abfd-dae5daab91ef-federate-client-tls\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.990941 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.990777    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4dce54-5199-40ce-abfd-dae5daab91ef-serving-certs-ca-bundle\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.990941 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.990818    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c4dce54-5199-40ce-abfd-dae5daab91ef-secret-telemeter-client\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.991582 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.991555    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4dce54-5199-40ce-abfd-dae5daab91ef-serving-certs-ca-bundle\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.991673 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.991600    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4dce54-5199-40ce-abfd-dae5daab91ef-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.991889 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.991871    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c4dce54-5199-40ce-abfd-dae5daab91ef-metrics-client-ca\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.993734 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.993710    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c4dce54-5199-40ce-abfd-dae5daab91ef-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.993835 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.993819    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c4dce54-5199-40ce-abfd-dae5daab91ef-telemeter-client-tls\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.993975 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.993954    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c4dce54-5199-40ce-abfd-dae5daab91ef-secret-telemeter-client\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:07.994021 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:07.993954    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c4dce54-5199-40ce-abfd-dae5daab91ef-federate-client-tls\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:08.052609 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:08.052581    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvrcq\" (UniqueName: \"kubernetes.io/projected/9c4dce54-5199-40ce-abfd-dae5daab91ef-kube-api-access-kvrcq\") pod \"telemeter-client-8b8f5df56-mxb7w\" (UID: \"9c4dce54-5199-40ce-abfd-dae5daab91ef\") " pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:08.332578 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:08.332546    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"
Nov 04 03:19:08.405068 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:08.404616    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1","Type":"ContainerStarted","Data":"e0151f44423b6edfe6aa139fd393fe64e3c71f77e074c4edd3eaf654d54aac5b"}
Nov 04 03:19:08.405068 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:08.404700    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1","Type":"ContainerStarted","Data":"2cbf466f5dae82273680c36c0ce37a9a5f674aad250b9a639d7e46c800d28c67"}
Nov 04 03:19:08.405068 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:08.404742    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1","Type":"ContainerStarted","Data":"6415f009190a195ade001c0fdbd64f8d93a56caeb1ae33f27b5804289307a2fa"}
Nov 04 03:19:08.405068 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:08.404782    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1","Type":"ContainerStarted","Data":"ffa543c0e408a124b4f5c7c1ffde486fbec51dd976198e7f321793d2fa5ba559"}
Nov 04 03:19:08.405068 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:08.404818    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1","Type":"ContainerStarted","Data":"520780e16617449e8e5ba94091824569d24c5ebaeccd9c6e32da8c8735cd74ae"}
Nov 04 03:19:08.405068 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:08.404879    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fcfc254d-52a4-4beb-8772-d2ba0ba39cd1","Type":"ContainerStarted","Data":"8fcd3eb647980ae2eb3625ca0f23a6538b806d1d8c28b90890d0d37bbd3a186e"}
Nov 04 03:19:08.536116 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:08.536062    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.521173525 podStartE2EDuration="3.536047597s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.874072ms" isStatefulPod=false podCreationTimestamp="2026-11-04 03:19:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:19:08.535032668 +0000 UTC m=+114.739632991" watchObservedRunningTime="2026-11-04 03:19:08.536047597 +0000 UTC m=+114.740647898"
Nov 04 03:19:08.757986 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:08.757834    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8b8f5df56-mxb7w"]
Nov 04 03:19:08.760979 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:19:08.760951    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c4dce54_5199_40ce_abfd_dae5daab91ef.slice/crio-5b00ba11ee8bf7844647fb92eed4ec54fdaba3e7a8c2990d147b7407034307a6 WatchSource:0}: Error finding container 5b00ba11ee8bf7844647fb92eed4ec54fdaba3e7a8c2990d147b7407034307a6: Status 404 returned error can't find the container with id 5b00ba11ee8bf7844647fb92eed4ec54fdaba3e7a8c2990d147b7407034307a6
Nov 04 03:19:09.410349 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:09.410310    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w" event={"ID":"9c4dce54-5199-40ce-abfd-dae5daab91ef","Type":"ContainerStarted","Data":"5b00ba11ee8bf7844647fb92eed4ec54fdaba3e7a8c2990d147b7407034307a6"}
Nov 04 03:19:11.421210 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:11.421125    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w" event={"ID":"9c4dce54-5199-40ce-abfd-dae5daab91ef","Type":"ContainerStarted","Data":"fce60554984f5087e2ff9be175c6e7f8e0d76afee951261e93728d2d263188e4"}
Nov 04 03:19:11.421210 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:11.421183    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w" event={"ID":"9c4dce54-5199-40ce-abfd-dae5daab91ef","Type":"ContainerStarted","Data":"c2f1ce80b2ce595260f33f8f1571eefcb4c333e25e321c2d260a5df673934c5a"}
Nov 04 03:19:11.421613 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:11.421210    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w" event={"ID":"9c4dce54-5199-40ce-abfd-dae5daab91ef","Type":"ContainerStarted","Data":"5ecdb76d718db85cb80cc8d7f16bdc425fe8f1992444827630a63be2e54c75d8"}
Nov 04 03:19:11.694281 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:11.694180    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-8b8f5df56-mxb7w" podStartSLOduration=2.3487806 podStartE2EDuration="4.694165243s" totalImagesPullingTime="2.345384643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:19:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:19:11.643869402 +0000 UTC m=+117.848469704" watchObservedRunningTime="2026-11-04 03:19:11.694165243 +0000 UTC m=+117.898765545"
Nov 04 03:19:12.084581 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.084546    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-9fdddb8d-swxwx"]
Nov 04 03:19:12.087134 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.087113    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.122781 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.122748    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4sbn\" (UniqueName: \"kubernetes.io/projected/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-kube-api-access-w4sbn\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.122781 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.122784    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-oauth-config\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.122995 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.122800    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-config\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.122995 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.122883    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-tmp\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.122995 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.122981    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-serving-cert\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.123108 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.123013    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-trusted-ca-bundle\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.123108 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.123032    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-service-ca\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.123108 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.123050    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-oauth-serving-cert\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.148036 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.148011    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9fdddb8d-swxwx"]
Nov 04 03:19:12.224285 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.224249    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-serving-cert\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.224449 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.224293    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-trusted-ca-bundle\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.224449 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.224337    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-service-ca\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.224449 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.224361    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-oauth-serving-cert\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.224449 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.224389    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4sbn\" (UniqueName: \"kubernetes.io/projected/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-kube-api-access-w4sbn\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.224449 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.224419    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-oauth-config\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.224449 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.224443    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-config\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.224765 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.224483    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-tmp\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.225006 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.224986    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-tmp\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.225246 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.225222    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-oauth-serving-cert\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.225307 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.225246    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-config\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.225307 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.225294    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-service-ca\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.225628 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.225605    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-trusted-ca-bundle\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.226924 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.226898    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-serving-cert\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.227046 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.227030    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-oauth-config\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.285667 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.285641    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4sbn\" (UniqueName: \"kubernetes.io/projected/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-kube-api-access-w4sbn\") pod \"console-9fdddb8d-swxwx\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") " pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.398152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.398073    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:12.778418 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:12.778385    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9fdddb8d-swxwx"]
Nov 04 03:19:12.782368 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:19:12.782343    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa01da7d_ba70_4599_9823_ee7bf63d2fa7.slice/crio-803d58338f162d7ae36c7855562f4f8966b07c7fcf71c86ab28751342146078a WatchSource:0}: Error finding container 803d58338f162d7ae36c7855562f4f8966b07c7fcf71c86ab28751342146078a: Status 404 returned error can't find the container with id 803d58338f162d7ae36c7855562f4f8966b07c7fcf71c86ab28751342146078a
Nov 04 03:19:13.429939 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:13.429902    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9fdddb8d-swxwx" event={"ID":"fa01da7d-ba70-4599-9823-ee7bf63d2fa7","Type":"ContainerStarted","Data":"56006329530a16396f2180fe801aabe54d6185d79d04bb115923cc39adcf9a83"}
Nov 04 03:19:13.430117 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:13.429963    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9fdddb8d-swxwx" event={"ID":"fa01da7d-ba70-4599-9823-ee7bf63d2fa7","Type":"ContainerStarted","Data":"803d58338f162d7ae36c7855562f4f8966b07c7fcf71c86ab28751342146078a"}
Nov 04 03:19:13.599639 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:13.599586    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-9fdddb8d-swxwx" podStartSLOduration=1.599569521 podStartE2EDuration="1.599569521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:19:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:19:13.548547003 +0000 UTC m=+119.753147323" watchObservedRunningTime="2026-11-04 03:19:13.599569521 +0000 UTC m=+119.804169888"
Nov 04 03:19:22.398590 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:22.398555    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:22.399029 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:22.398619    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:22.403448 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:22.403423    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:22.466449 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:22.466423    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:19:22.673458 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:22.673372    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f4dc574c4-72n8l"]
Nov 04 03:19:28.728376 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:19:28.728297    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:19:47.692645 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:47.692606    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6f4dc574c4-72n8l" podUID="b99e5303-fbd3-44d8-8bb0-a4db17ab80f0" containerName="console" containerID="cri-o://ddd5477a9b88c17281976e7da09c1fe1d0bb6db9b0ba804006b836d67e318df8" gracePeriod=15
Nov 04 03:19:47.952938 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:47.952881    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:19:48.019318 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.019286    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-kube-api-access-hp8x6\" (UniqueName: \"kubernetes.io/projected/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-kube-api-access-hp8x6\") pod \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") "
Nov 04 03:19:48.019470 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.019336    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-serving-cert\") pod \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") "
Nov 04 03:19:48.019470 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.019357    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-oauth-serving-cert\") pod \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") "
Nov 04 03:19:48.019470 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.019390    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-trusted-ca-bundle\") pod \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") "
Nov 04 03:19:48.019470 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.019424    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-service-ca\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-service-ca\") pod \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") "
Nov 04 03:19:48.019470 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.019465    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-config\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-config\") pod \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") "
Nov 04 03:19:48.019743 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.019612    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-oauth-config\") pod \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") "
Nov 04 03:19:48.019891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.019873    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-tmp\") pod \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\" (UID: \"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0\") "
Nov 04 03:19:48.020163 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.019895    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-service-ca" pod "b99e5303-fbd3-44d8-8bb0-a4db17ab80f0" (UID: "b99e5303-fbd3-44d8-8bb0-a4db17ab80f0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:19:48.020218 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.019900    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-oauth-serving-cert" pod "b99e5303-fbd3-44d8-8bb0-a4db17ab80f0" (UID: "b99e5303-fbd3-44d8-8bb0-a4db17ab80f0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:19:48.020218 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.019908    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-config" pod "b99e5303-fbd3-44d8-8bb0-a4db17ab80f0" (UID: "b99e5303-fbd3-44d8-8bb0-a4db17ab80f0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:19:48.020218 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.019914    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-trusted-ca-bundle" pod "b99e5303-fbd3-44d8-8bb0-a4db17ab80f0" (UID: "b99e5303-fbd3-44d8-8bb0-a4db17ab80f0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:19:48.020218 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.020181    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-tmp" pod "b99e5303-fbd3-44d8-8bb0-a4db17ab80f0" (UID: "b99e5303-fbd3-44d8-8bb0-a4db17ab80f0"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:19:48.020484 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.020464    4521 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-oauth-serving-cert\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:48.020525 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.020490    4521 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-trusted-ca-bundle\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:48.020525 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.020504    4521 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-service-ca\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:48.020525 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.020518    4521 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-config\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:48.020611 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.020532    4521 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-tmp\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:48.021603 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.021574    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-serving-cert" pod "b99e5303-fbd3-44d8-8bb0-a4db17ab80f0" (UID: "b99e5303-fbd3-44d8-8bb0-a4db17ab80f0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:48.021694 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.021599    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-kube-api-access-hp8x6" pod "b99e5303-fbd3-44d8-8bb0-a4db17ab80f0" (UID: "b99e5303-fbd3-44d8-8bb0-a4db17ab80f0"). InnerVolumeSpecName "kube-api-access-hp8x6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:19:48.021694 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.021683    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-oauth-config" pod "b99e5303-fbd3-44d8-8bb0-a4db17ab80f0" (UID: "b99e5303-fbd3-44d8-8bb0-a4db17ab80f0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:48.121287 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.121254    4521 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-oauth-config\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:48.121287 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.121280    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hp8x6\" (UniqueName: \"kubernetes.io/projected/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-kube-api-access-hp8x6\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:48.121287 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.121291    4521 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0-console-serving-cert\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:48.570076 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.570040    4521 generic.go:356] "Generic (PLEG): container finished" podID="b99e5303-fbd3-44d8-8bb0-a4db17ab80f0" containerID="ddd5477a9b88c17281976e7da09c1fe1d0bb6db9b0ba804006b836d67e318df8" exitCode=0
Nov 04 03:19:48.570243 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.570102    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f4dc574c4-72n8l"
Nov 04 03:19:48.570243 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.570113    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f4dc574c4-72n8l" event={"ID":"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0","Type":"ContainerDied","Data":"ddd5477a9b88c17281976e7da09c1fe1d0bb6db9b0ba804006b836d67e318df8"}
Nov 04 03:19:48.570243 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.570172    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f4dc574c4-72n8l" event={"ID":"b99e5303-fbd3-44d8-8bb0-a4db17ab80f0","Type":"ContainerDied","Data":"b7cd1ce53b23872b33e94fd83e99e98f2f31bcea84084905ec1ee2136f2f4d21"}
Nov 04 03:19:48.570243 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.570193    4521 scope.go:122] "RemoveContainer" containerID="ddd5477a9b88c17281976e7da09c1fe1d0bb6db9b0ba804006b836d67e318df8"
Nov 04 03:19:48.578945 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.578925    4521 scope.go:122] "RemoveContainer" containerID="ddd5477a9b88c17281976e7da09c1fe1d0bb6db9b0ba804006b836d67e318df8"
Nov 04 03:19:48.579229 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:19:48.579209    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddd5477a9b88c17281976e7da09c1fe1d0bb6db9b0ba804006b836d67e318df8\": container with ID starting with ddd5477a9b88c17281976e7da09c1fe1d0bb6db9b0ba804006b836d67e318df8 not found: ID does not exist" containerID="ddd5477a9b88c17281976e7da09c1fe1d0bb6db9b0ba804006b836d67e318df8"
Nov 04 03:19:48.579281 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.579236    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddd5477a9b88c17281976e7da09c1fe1d0bb6db9b0ba804006b836d67e318df8"} err="failed to get container status \"ddd5477a9b88c17281976e7da09c1fe1d0bb6db9b0ba804006b836d67e318df8\": rpc error: code = NotFound desc = could not find container \"ddd5477a9b88c17281976e7da09c1fe1d0bb6db9b0ba804006b836d67e318df8\": container with ID starting with ddd5477a9b88c17281976e7da09c1fe1d0bb6db9b0ba804006b836d67e318df8 not found: ID does not exist"
Nov 04 03:19:48.735570 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.735535    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f4dc574c4-72n8l"]
Nov 04 03:19:48.785815 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:48.785785    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f4dc574c4-72n8l"]
Nov 04 03:19:49.909467 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:19:49.909435    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b99e5303-fbd3-44d8-8bb0-a4db17ab80f0" path="/var/lib/kubelet/pods/b99e5303-fbd3-44d8-8bb0-a4db17ab80f0/volumes"
Nov 04 03:19:58.728311 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:19:58.728268    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:20:26.143154 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.143122    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6575766b8-qrpvx"]
Nov 04 03:20:26.143795 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.143610    4521 cpu_manager.go:414] "containerMap: removing container" podUID="b99e5303-fbd3-44d8-8bb0-a4db17ab80f0" containerName="console"
Nov 04 03:20:26.143795 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.143631    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b99e5303-fbd3-44d8-8bb0-a4db17ab80f0" containerName="console"
Nov 04 03:20:26.143795 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.143716    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="b99e5303-fbd3-44d8-8bb0-a4db17ab80f0" containerName="console"
Nov 04 03:20:26.146652 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.146630    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.213674 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.213637    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20b96505-8ab2-4701-ad28-a986181e670b-console-config\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.213674 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.213679    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20b96505-8ab2-4701-ad28-a986181e670b-console-serving-cert\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.213902 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.213700    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gvfr\" (UniqueName: \"kubernetes.io/projected/20b96505-8ab2-4701-ad28-a986181e670b-kube-api-access-9gvfr\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.213902 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.213770    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20b96505-8ab2-4701-ad28-a986181e670b-tmp\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.213902 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.213806    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20b96505-8ab2-4701-ad28-a986181e670b-trusted-ca-bundle\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.213902 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.213884    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20b96505-8ab2-4701-ad28-a986181e670b-oauth-serving-cert\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.214025 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.213948    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20b96505-8ab2-4701-ad28-a986181e670b-console-oauth-config\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.214025 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.213971    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20b96505-8ab2-4701-ad28-a986181e670b-service-ca\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.250581 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.250551    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6575766b8-qrpvx"]
Nov 04 03:20:26.315277 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.315241    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20b96505-8ab2-4701-ad28-a986181e670b-tmp\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.315277 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.315281    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20b96505-8ab2-4701-ad28-a986181e670b-trusted-ca-bundle\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.315502 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.315320    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20b96505-8ab2-4701-ad28-a986181e670b-oauth-serving-cert\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.315502 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.315348    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20b96505-8ab2-4701-ad28-a986181e670b-console-oauth-config\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.315502 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.315369    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20b96505-8ab2-4701-ad28-a986181e670b-service-ca\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.315502 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.315418    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20b96505-8ab2-4701-ad28-a986181e670b-console-config\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.315502 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.315447    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20b96505-8ab2-4701-ad28-a986181e670b-console-serving-cert\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.315502 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.315464    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gvfr\" (UniqueName: \"kubernetes.io/projected/20b96505-8ab2-4701-ad28-a986181e670b-kube-api-access-9gvfr\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.315792 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.315755    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20b96505-8ab2-4701-ad28-a986181e670b-tmp\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.316192 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.316168    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20b96505-8ab2-4701-ad28-a986181e670b-oauth-serving-cert\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.316282 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.316184    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20b96505-8ab2-4701-ad28-a986181e670b-console-config\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.316321 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.316281    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20b96505-8ab2-4701-ad28-a986181e670b-service-ca\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.316637 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.316614    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20b96505-8ab2-4701-ad28-a986181e670b-trusted-ca-bundle\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.317902 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.317878    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20b96505-8ab2-4701-ad28-a986181e670b-console-oauth-config\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.317982 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.317965    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20b96505-8ab2-4701-ad28-a986181e670b-console-serving-cert\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.376918 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.376887    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gvfr\" (UniqueName: \"kubernetes.io/projected/20b96505-8ab2-4701-ad28-a986181e670b-kube-api-access-9gvfr\") pod \"console-6575766b8-qrpvx\" (UID: \"20b96505-8ab2-4701-ad28-a986181e670b\") " pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.458784 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.458703    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:26.834873 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:26.834670    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6575766b8-qrpvx"]
Nov 04 03:20:26.837553 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:20:26.837521    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20b96505_8ab2_4701_ad28_a986181e670b.slice/crio-2725963f450b2d5c728ada62731ed664c1b26758099c24902c99b23683f0f643 WatchSource:0}: Error finding container 2725963f450b2d5c728ada62731ed664c1b26758099c24902c99b23683f0f643: Status 404 returned error can't find the container with id 2725963f450b2d5c728ada62731ed664c1b26758099c24902c99b23683f0f643
Nov 04 03:20:27.709235 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:27.709196    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6575766b8-qrpvx" event={"ID":"20b96505-8ab2-4701-ad28-a986181e670b","Type":"ContainerStarted","Data":"fa8d0dfa7b5b75934f215a4df5e8c785db7d9b1a58651529de488a740f9fe79e"}
Nov 04 03:20:27.709626 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:27.709248    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6575766b8-qrpvx" event={"ID":"20b96505-8ab2-4701-ad28-a986181e670b","Type":"ContainerStarted","Data":"2725963f450b2d5c728ada62731ed664c1b26758099c24902c99b23683f0f643"}
Nov 04 03:20:27.828687 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:27.828633    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6575766b8-qrpvx" podStartSLOduration=1.828618789 podStartE2EDuration="1.828618789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:20:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:20:27.827807969 +0000 UTC m=+194.032408304" watchObservedRunningTime="2026-11-04 03:20:27.828618789 +0000 UTC m=+194.033219091"
Nov 04 03:20:28.728946 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:20:28.728911    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:20:36.459712 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:36.459660    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:36.460244 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:36.459749    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:36.464552 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:36.464528    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:36.744176 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:36.744094    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6575766b8-qrpvx"
Nov 04 03:20:36.897776 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:20:36.897742    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9fdddb8d-swxwx"]
Nov 04 03:20:58.729149 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:20:58.729063    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:21:01.916777 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:01.916733    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-9fdddb8d-swxwx" podUID="fa01da7d-ba70-4599-9823-ee7bf63d2fa7" containerName="console" containerID="cri-o://56006329530a16396f2180fe801aabe54d6185d79d04bb115923cc39adcf9a83" gracePeriod=15
Nov 04 03:21:02.157059 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.157035    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:21:02.322039 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.322006    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-serving-cert\") pod \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") "
Nov 04 03:21:02.322205 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.322078    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-oauth-config\") pod \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") "
Nov 04 03:21:02.322205 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.322134    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-service-ca\") pod \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") "
Nov 04 03:21:02.322205 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.322166    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-kube-api-access-w4sbn\" (UniqueName: \"kubernetes.io/projected/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-kube-api-access-w4sbn\") pod \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") "
Nov 04 03:21:02.322205 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.322192    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-tmp\") pod \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") "
Nov 04 03:21:02.322430 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.322220    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-config\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-config\") pod \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") "
Nov 04 03:21:02.322430 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.322247    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-oauth-serving-cert\") pod \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") "
Nov 04 03:21:02.322430 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.322283    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-trusted-ca-bundle\") pod \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\" (UID: \"fa01da7d-ba70-4599-9823-ee7bf63d2fa7\") "
Nov 04 03:21:02.322564 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.322533    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-tmp" pod "fa01da7d-ba70-4599-9823-ee7bf63d2fa7" (UID: "fa01da7d-ba70-4599-9823-ee7bf63d2fa7"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:21:02.322654 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.322630    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-config" pod "fa01da7d-ba70-4599-9823-ee7bf63d2fa7" (UID: "fa01da7d-ba70-4599-9823-ee7bf63d2fa7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:21:02.322714 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.322650    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-service-ca" pod "fa01da7d-ba70-4599-9823-ee7bf63d2fa7" (UID: "fa01da7d-ba70-4599-9823-ee7bf63d2fa7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:21:02.322881 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.322837    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-oauth-serving-cert" pod "fa01da7d-ba70-4599-9823-ee7bf63d2fa7" (UID: "fa01da7d-ba70-4599-9823-ee7bf63d2fa7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:21:02.323025 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.322883    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-trusted-ca-bundle" pod "fa01da7d-ba70-4599-9823-ee7bf63d2fa7" (UID: "fa01da7d-ba70-4599-9823-ee7bf63d2fa7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:21:02.324264 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.324243    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-oauth-config" pod "fa01da7d-ba70-4599-9823-ee7bf63d2fa7" (UID: "fa01da7d-ba70-4599-9823-ee7bf63d2fa7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:21:02.324710 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.324694    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-serving-cert" pod "fa01da7d-ba70-4599-9823-ee7bf63d2fa7" (UID: "fa01da7d-ba70-4599-9823-ee7bf63d2fa7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:21:02.324758 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.324723    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-kube-api-access-w4sbn" pod "fa01da7d-ba70-4599-9823-ee7bf63d2fa7" (UID: "fa01da7d-ba70-4599-9823-ee7bf63d2fa7"). InnerVolumeSpecName "kube-api-access-w4sbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:21:02.423624 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.423583    4521 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-serving-cert\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:21:02.423624 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.423618    4521 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-oauth-config\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:21:02.423624 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.423634    4521 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-service-ca\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:21:02.423941 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.423646    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4sbn\" (UniqueName: \"kubernetes.io/projected/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-kube-api-access-w4sbn\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:21:02.427787 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.423806    4521 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-tmp\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:21:02.427787 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.424103    4521 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-console-config\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:21:02.427787 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.424121    4521 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-oauth-serving-cert\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:21:02.427787 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.424138    4521 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa01da7d-ba70-4599-9823-ee7bf63d2fa7-trusted-ca-bundle\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:21:02.835889 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.835836    4521 generic.go:356] "Generic (PLEG): container finished" podID="fa01da7d-ba70-4599-9823-ee7bf63d2fa7" containerID="56006329530a16396f2180fe801aabe54d6185d79d04bb115923cc39adcf9a83" exitCode=0
Nov 04 03:21:02.836080 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.835899    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9fdddb8d-swxwx" event={"ID":"fa01da7d-ba70-4599-9823-ee7bf63d2fa7","Type":"ContainerDied","Data":"56006329530a16396f2180fe801aabe54d6185d79d04bb115923cc39adcf9a83"}
Nov 04 03:21:02.836080 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.835917    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9fdddb8d-swxwx"
Nov 04 03:21:02.836080 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.835967    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9fdddb8d-swxwx" event={"ID":"fa01da7d-ba70-4599-9823-ee7bf63d2fa7","Type":"ContainerDied","Data":"803d58338f162d7ae36c7855562f4f8966b07c7fcf71c86ab28751342146078a"}
Nov 04 03:21:02.836080 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.835989    4521 scope.go:122] "RemoveContainer" containerID="56006329530a16396f2180fe801aabe54d6185d79d04bb115923cc39adcf9a83"
Nov 04 03:21:02.844381 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.844361    4521 scope.go:122] "RemoveContainer" containerID="56006329530a16396f2180fe801aabe54d6185d79d04bb115923cc39adcf9a83"
Nov 04 03:21:02.844661 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:21:02.844642    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56006329530a16396f2180fe801aabe54d6185d79d04bb115923cc39adcf9a83\": container with ID starting with 56006329530a16396f2180fe801aabe54d6185d79d04bb115923cc39adcf9a83 not found: ID does not exist" containerID="56006329530a16396f2180fe801aabe54d6185d79d04bb115923cc39adcf9a83"
Nov 04 03:21:02.844721 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:02.844672    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56006329530a16396f2180fe801aabe54d6185d79d04bb115923cc39adcf9a83"} err="failed to get container status \"56006329530a16396f2180fe801aabe54d6185d79d04bb115923cc39adcf9a83\": rpc error: code = NotFound desc = could not find container \"56006329530a16396f2180fe801aabe54d6185d79d04bb115923cc39adcf9a83\": container with ID starting with 56006329530a16396f2180fe801aabe54d6185d79d04bb115923cc39adcf9a83 not found: ID does not exist"
Nov 04 03:21:03.000549 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:03.000512    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9fdddb8d-swxwx"]
Nov 04 03:21:03.050933 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:03.050901    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9fdddb8d-swxwx"]
Nov 04 03:21:03.908911 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:21:03.908879    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa01da7d-ba70-4599-9823-ee7bf63d2fa7" path="/var/lib/kubelet/pods/fa01da7d-ba70-4599-9823-ee7bf63d2fa7/volumes"
Nov 04 03:21:28.728488 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:21:28.728444    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:21:58.728209 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:21:58.728166    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:22:28.729170 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:22:28.729085    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:22:51.759586 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.759552    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"]
Nov 04 03:22:51.760071 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.759894    4521 cpu_manager.go:414] "containerMap: removing container" podUID="fa01da7d-ba70-4599-9823-ee7bf63d2fa7" containerName="console"
Nov 04 03:22:51.760071 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.759909    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa01da7d-ba70-4599-9823-ee7bf63d2fa7" containerName="console"
Nov 04 03:22:51.760071 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.759973    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa01da7d-ba70-4599-9823-ee7bf63d2fa7" containerName="console"
Nov 04 03:22:51.762933 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.762917    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.809957 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.809925    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.810088 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.809969    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vnxm\" (UniqueName: \"kubernetes.io/projected/a5a024f1-d038-44c2-bf29-6d3365192f7f-kube-api-access-2vnxm\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.810088 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.810040    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.810168 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.810108    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.810168 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.810133    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5a024f1-d038-44c2-bf29-6d3365192f7f-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.810168 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.810154    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.810276 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.810216    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.817626 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.817591    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 04 03:22:51.817626 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.817602    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 03:22:51.817882 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.817787    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 04 03:22:51.817882 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.817812    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 04 03:22:51.817882 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.817821    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 03:22:51.817882 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.817815    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 04 03:22:51.868248 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.868220    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-868th\""
Nov 04 03:22:51.869312 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.869291    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"]
Nov 04 03:22:51.910469 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.910437    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.910639 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.910498    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.910639 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.910529    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5a024f1-d038-44c2-bf29-6d3365192f7f-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.910639 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.910552    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.910639 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.910599    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.910865 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.910745    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.910865 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.910831    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vnxm\" (UniqueName: \"kubernetes.io/projected/a5a024f1-d038-44c2-bf29-6d3365192f7f-kube-api-access-2vnxm\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.911280 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.911256    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.912806 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.912787    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5a024f1-d038-44c2-bf29-6d3365192f7f-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.913107 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.913086    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.913210 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.913194    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.913310 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.913291    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.971903 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.971874    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vnxm\" (UniqueName: \"kubernetes.io/projected/a5a024f1-d038-44c2-bf29-6d3365192f7f-kube-api-access-2vnxm\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:51.972033 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:51.971915    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-mzwml\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:52.073622 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:52.073595    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:52.496887 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:52.496777    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"]
Nov 04 03:22:52.503728 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:22:52.503703    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5a024f1_d038_44c2_bf29_6d3365192f7f.slice/crio-fb1433f0b9f54785e697a6a4bf23412fef9743103d91034e5a8e8004771f133e WatchSource:0}: Error finding container fb1433f0b9f54785e697a6a4bf23412fef9743103d91034e5a8e8004771f133e: Status 404 returned error can't find the container with id fb1433f0b9f54785e697a6a4bf23412fef9743103d91034e5a8e8004771f133e
Nov 04 03:22:52.505536 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:52.505519    4521 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:22:53.218199 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:53.218160    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml" event={"ID":"a5a024f1-d038-44c2-bf29-6d3365192f7f","Type":"ContainerStarted","Data":"fb1433f0b9f54785e697a6a4bf23412fef9743103d91034e5a8e8004771f133e"}
Nov 04 03:22:56.836323 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:56.836289    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:22:56.836618 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:56.836354    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:22:57.235764 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:57.235678    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml" event={"ID":"a5a024f1-d038-44c2-bf29-6d3365192f7f","Type":"ContainerStarted","Data":"9c705c8599a2a50718f43f6d5ad31b4595c883bbe469ee339343acc1a8c3928f"}
Nov 04 03:22:57.235968 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:57.235878    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:22:57.237385 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:57.237356    4521 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-mzwml container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:22:57.237492 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:57.237405    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml" podUID="a5a024f1-d038-44c2-bf29-6d3365192f7f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:22:57.445191 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:57.445145    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml" podStartSLOduration=2.115369206 podStartE2EDuration="6.445131508s" totalImagesPullingTime="4.329762302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:22:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:22:57.407257285 +0000 UTC m=+343.611857605" watchObservedRunningTime="2026-11-04 03:22:57.445131508 +0000 UTC m=+343.649731809"
Nov 04 03:22:58.242607 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:58.242551    4521 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-mzwml container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:22:58.243274 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:22:58.242615    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml" podUID="a5a024f1-d038-44c2-bf29-6d3365192f7f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:22:58.729020 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:22:58.728985    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:23:01.243096 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:01.243061    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:23:13.926482 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:13.926452    4521 scope.go:122] "RemoveContainer" containerID="73e9e632fe2a67086b6eb08e1342e7886c729089d953f698fc398261e8fc080c"
Nov 04 03:23:28.728385 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:23:28.728344    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:23:46.618883 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.618832    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"]
Nov 04 03:23:46.621921 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.621905    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.680143 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.680106    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fe4eb27f-b746-4648-a62e-c33105e65917-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.680347 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.680173    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fe4eb27f-b746-4648-a62e-c33105e65917-manifests\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.680347 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.680251    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fe4eb27f-b746-4648-a62e-c33105e65917-metrics-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.680347 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.680283    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe4eb27f-b746-4648-a62e-c33105e65917-webhook-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.680347 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.680303    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6865l\" (UniqueName: \"kubernetes.io/projected/fe4eb27f-b746-4648-a62e-c33105e65917-kube-api-access-6865l\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.680605 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.680406    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe4eb27f-b746-4648-a62e-c33105e65917-tmp\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.726299 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.726262    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 04 03:23:46.726476 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.726303    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 04 03:23:46.727455 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.727437    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"]
Nov 04 03:23:46.754998 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.754965    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 04 03:23:46.755186 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.755093    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 04 03:23:46.755251 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.755187    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-9kg9c\""
Nov 04 03:23:46.755251 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.755219    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 04 03:23:46.781037 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.781006    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fe4eb27f-b746-4648-a62e-c33105e65917-manifests\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.781037 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.781038    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fe4eb27f-b746-4648-a62e-c33105e65917-metrics-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.781285 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.781059    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe4eb27f-b746-4648-a62e-c33105e65917-webhook-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.781285 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.781081    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6865l\" (UniqueName: \"kubernetes.io/projected/fe4eb27f-b746-4648-a62e-c33105e65917-kube-api-access-6865l\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.781285 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.781116    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe4eb27f-b746-4648-a62e-c33105e65917-tmp\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.781285 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.781181    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fe4eb27f-b746-4648-a62e-c33105e65917-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.781660 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.781508    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fe4eb27f-b746-4648-a62e-c33105e65917-manifests\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.783532 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.783503    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe4eb27f-b746-4648-a62e-c33105e65917-tmp\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.783669 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.783649    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe4eb27f-b746-4648-a62e-c33105e65917-webhook-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.783949 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.783932    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fe4eb27f-b746-4648-a62e-c33105e65917-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.783993 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.783964    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fe4eb27f-b746-4648-a62e-c33105e65917-metrics-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.847546 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.847515    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6865l\" (UniqueName: \"kubernetes.io/projected/fe4eb27f-b746-4648-a62e-c33105e65917-kube-api-access-6865l\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-5cfww\" (UID: \"fe4eb27f-b746-4648-a62e-c33105e65917\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:46.937266 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:46.937181    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:47.433020 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:47.432983    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"]
Nov 04 03:23:47.456977 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:23:47.456947    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe4eb27f_b746_4648_a62e_c33105e65917.slice/crio-81dd9eef5743f7961bc05734ff3814c8424e1689a9a71cb95f9cd9a89feee1cd WatchSource:0}: Error finding container 81dd9eef5743f7961bc05734ff3814c8424e1689a9a71cb95f9cd9a89feee1cd: Status 404 returned error can't find the container with id 81dd9eef5743f7961bc05734ff3814c8424e1689a9a71cb95f9cd9a89feee1cd
Nov 04 03:23:48.419148 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:48.419105    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww" event={"ID":"fe4eb27f-b746-4648-a62e-c33105e65917","Type":"ContainerStarted","Data":"81dd9eef5743f7961bc05734ff3814c8424e1689a9a71cb95f9cd9a89feee1cd"}
Nov 04 03:23:52.441171 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:52.441136    4521 generic.go:356] "Generic (PLEG): container finished" podID="fe4eb27f-b746-4648-a62e-c33105e65917" containerID="a1ee13b189cd19c70a2040309e4137cd8f893205e6f93d3833aefaad4586bc9a" exitCode=0
Nov 04 03:23:52.441616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:52.441206    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww" event={"ID":"fe4eb27f-b746-4648-a62e-c33105e65917","Type":"ContainerDied","Data":"a1ee13b189cd19c70a2040309e4137cd8f893205e6f93d3833aefaad4586bc9a"}
Nov 04 03:23:53.446950 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:53.446911    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww" event={"ID":"fe4eb27f-b746-4648-a62e-c33105e65917","Type":"ContainerStarted","Data":"2f35c193f14f2b8a560b314427ffe7f6fd866f3f52afed07cf8da5cf4fa4d2aa"}
Nov 04 03:23:53.447364 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:53.447051    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:23:53.649461 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:23:53.649349    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww" podStartSLOduration=1.991791221 podStartE2EDuration="7.649330045s" totalImagesPullingTime="5.187952182s" totalInitContainerRuntime="469.586642ms" isStatefulPod=false podCreationTimestamp="2026-11-04 03:23:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:23:53.647992585 +0000 UTC m=+399.852592907" watchObservedRunningTime="2026-11-04 03:23:53.649330045 +0000 UTC m=+399.853930345"
Nov 04 03:23:58.728716 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:23:58.728631    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:24:04.453600 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:04.453565    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-5cfww"
Nov 04 03:24:19.899609 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:19.899572    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-hb9vr"]
Nov 04 03:24:19.910371 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:19.910342    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hb9vr"
Nov 04 03:24:19.961121 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:19.961084    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-hb9vr"]
Nov 04 03:24:20.010628 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:20.010590    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 04 03:24:20.010791 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:20.010650    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 04 03:24:20.010791 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:20.010672    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cfrc6\""
Nov 04 03:24:20.010791 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:20.010673    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Nov 04 03:24:20.013702 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:20.013687    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Nov 04 03:24:20.076412 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:20.076383    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bd98323d-8102-4f46-8765-c7ee20226eb7-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-hb9vr\" (UID: \"bd98323d-8102-4f46-8765-c7ee20226eb7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hb9vr"
Nov 04 03:24:20.076561 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:20.076431    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd98323d-8102-4f46-8765-c7ee20226eb7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-hb9vr\" (UID: \"bd98323d-8102-4f46-8765-c7ee20226eb7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hb9vr"
Nov 04 03:24:20.076561 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:20.076452    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cflqb\" (UniqueName: \"kubernetes.io/projected/bd98323d-8102-4f46-8765-c7ee20226eb7-kube-api-access-cflqb\") pod \"kuadrant-console-plugin-66c4fb575d-hb9vr\" (UID: \"bd98323d-8102-4f46-8765-c7ee20226eb7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hb9vr"
Nov 04 03:24:20.177023 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:20.176939    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd98323d-8102-4f46-8765-c7ee20226eb7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-hb9vr\" (UID: \"bd98323d-8102-4f46-8765-c7ee20226eb7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hb9vr"
Nov 04 03:24:20.177023 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:20.176977    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cflqb\" (UniqueName: \"kubernetes.io/projected/bd98323d-8102-4f46-8765-c7ee20226eb7-kube-api-access-cflqb\") pod \"kuadrant-console-plugin-66c4fb575d-hb9vr\" (UID: \"bd98323d-8102-4f46-8765-c7ee20226eb7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hb9vr"
Nov 04 03:24:20.177203 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:20.177034    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bd98323d-8102-4f46-8765-c7ee20226eb7-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-hb9vr\" (UID: \"bd98323d-8102-4f46-8765-c7ee20226eb7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hb9vr"
Nov 04 03:24:20.177629 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:20.177605    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bd98323d-8102-4f46-8765-c7ee20226eb7-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-hb9vr\" (UID: \"bd98323d-8102-4f46-8765-c7ee20226eb7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hb9vr"
Nov 04 03:24:20.179437 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:20.179408    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd98323d-8102-4f46-8765-c7ee20226eb7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-hb9vr\" (UID: \"bd98323d-8102-4f46-8765-c7ee20226eb7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hb9vr"
Nov 04 03:24:20.239005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:20.238970    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cflqb\" (UniqueName: \"kubernetes.io/projected/bd98323d-8102-4f46-8765-c7ee20226eb7-kube-api-access-cflqb\") pod \"kuadrant-console-plugin-66c4fb575d-hb9vr\" (UID: \"bd98323d-8102-4f46-8765-c7ee20226eb7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hb9vr"
Nov 04 03:24:20.537934 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:20.537910    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hb9vr"
Nov 04 03:24:20.908006 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:20.907708    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-hb9vr"]
Nov 04 03:24:20.911184 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:24:20.911158    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd98323d_8102_4f46_8765_c7ee20226eb7.slice/crio-113def56ad0e3f0a37373d9b566870de6b5d9c98f751c0a8cc557ecc5d9353d4 WatchSource:0}: Error finding container 113def56ad0e3f0a37373d9b566870de6b5d9c98f751c0a8cc557ecc5d9353d4: Status 404 returned error can't find the container with id 113def56ad0e3f0a37373d9b566870de6b5d9c98f751c0a8cc557ecc5d9353d4
Nov 04 03:24:21.556999 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:21.556966    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hb9vr" event={"ID":"bd98323d-8102-4f46-8765-c7ee20226eb7","Type":"ContainerStarted","Data":"113def56ad0e3f0a37373d9b566870de6b5d9c98f751c0a8cc557ecc5d9353d4"}
Nov 04 03:24:27.584068 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:27.584028    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hb9vr" event={"ID":"bd98323d-8102-4f46-8765-c7ee20226eb7","Type":"ContainerStarted","Data":"131657ff8807b22ef3a4a5fe30cc53878a344ec71504e4f147985d3a2630e882"}
Nov 04 03:24:27.714888 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:27.714803    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hb9vr" podStartSLOduration=2.7717150249999998 podStartE2EDuration="8.714788723s" totalImagesPullingTime="5.943073698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:24:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:24:27.714591953 +0000 UTC m=+433.919192256" watchObservedRunningTime="2026-11-04 03:24:27.714788723 +0000 UTC m=+433.919389074"
Nov 04 03:24:28.729658 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:24:28.729620    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:24:35.305923 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:35.305892    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7659c7d778-65h9l"]
Nov 04 03:24:35.310891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:35.310871    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7659c7d778-65h9l"
Nov 04 03:24:35.382332 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:35.382295    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 04 03:24:35.382500 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:35.382412    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 04 03:24:35.390048 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:35.390014    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da5d1d62-7e67-423b-95ac-8c809a64959c-tmp\") pod \"kube-auth-proxy-7659c7d778-65h9l\" (UID: \"da5d1d62-7e67-423b-95ac-8c809a64959c\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-65h9l"
Nov 04 03:24:35.390209 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:35.390072    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sqp7\" (UniqueName: \"kubernetes.io/projected/da5d1d62-7e67-423b-95ac-8c809a64959c-kube-api-access-2sqp7\") pod \"kube-auth-proxy-7659c7d778-65h9l\" (UID: \"da5d1d62-7e67-423b-95ac-8c809a64959c\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-65h9l"
Nov 04 03:24:35.390292 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:35.390223    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da5d1d62-7e67-423b-95ac-8c809a64959c-tls-certs\") pod \"kube-auth-proxy-7659c7d778-65h9l\" (UID: \"da5d1d62-7e67-423b-95ac-8c809a64959c\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-65h9l"
Nov 04 03:24:35.414448 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:35.414424    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-sgt86\""
Nov 04 03:24:35.415562 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:35.415538    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7659c7d778-65h9l"]
Nov 04 03:24:35.491457 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:35.491426    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da5d1d62-7e67-423b-95ac-8c809a64959c-tls-certs\") pod \"kube-auth-proxy-7659c7d778-65h9l\" (UID: \"da5d1d62-7e67-423b-95ac-8c809a64959c\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-65h9l"
Nov 04 03:24:35.491616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:35.491478    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da5d1d62-7e67-423b-95ac-8c809a64959c-tmp\") pod \"kube-auth-proxy-7659c7d778-65h9l\" (UID: \"da5d1d62-7e67-423b-95ac-8c809a64959c\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-65h9l"
Nov 04 03:24:35.491616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:35.491512    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sqp7\" (UniqueName: \"kubernetes.io/projected/da5d1d62-7e67-423b-95ac-8c809a64959c-kube-api-access-2sqp7\") pod \"kube-auth-proxy-7659c7d778-65h9l\" (UID: \"da5d1d62-7e67-423b-95ac-8c809a64959c\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-65h9l"
Nov 04 03:24:35.493914 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:35.493880    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da5d1d62-7e67-423b-95ac-8c809a64959c-tmp\") pod \"kube-auth-proxy-7659c7d778-65h9l\" (UID: \"da5d1d62-7e67-423b-95ac-8c809a64959c\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-65h9l"
Nov 04 03:24:35.494070 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:35.494050    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da5d1d62-7e67-423b-95ac-8c809a64959c-tls-certs\") pod \"kube-auth-proxy-7659c7d778-65h9l\" (UID: \"da5d1d62-7e67-423b-95ac-8c809a64959c\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-65h9l"
Nov 04 03:24:35.553395 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:35.553363    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sqp7\" (UniqueName: \"kubernetes.io/projected/da5d1d62-7e67-423b-95ac-8c809a64959c-kube-api-access-2sqp7\") pod \"kube-auth-proxy-7659c7d778-65h9l\" (UID: \"da5d1d62-7e67-423b-95ac-8c809a64959c\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-65h9l"
Nov 04 03:24:35.622905 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:35.622819    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7659c7d778-65h9l"
Nov 04 03:24:35.995948 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:35.995772    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7659c7d778-65h9l"]
Nov 04 03:24:35.998714 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:24:35.998689    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda5d1d62_7e67_423b_95ac_8c809a64959c.slice/crio-2f95297a9ec58a431df459aba0eb1f4431dd3b9dcb047279448aa4cc32eeb048 WatchSource:0}: Error finding container 2f95297a9ec58a431df459aba0eb1f4431dd3b9dcb047279448aa4cc32eeb048: Status 404 returned error can't find the container with id 2f95297a9ec58a431df459aba0eb1f4431dd3b9dcb047279448aa4cc32eeb048
Nov 04 03:24:36.623045 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:36.623009    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7659c7d778-65h9l" event={"ID":"da5d1d62-7e67-423b-95ac-8c809a64959c","Type":"ContainerStarted","Data":"2f95297a9ec58a431df459aba0eb1f4431dd3b9dcb047279448aa4cc32eeb048"}
Nov 04 03:24:38.881256 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:38.881214    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"]
Nov 04 03:24:38.892503 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:38.892480    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"
Nov 04 03:24:38.918405 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:38.918369    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d764ab92-e631-4c4c-a3fc-b71b86a69c33-cert\") pod \"jobset-controller-manager-696bfff7cc-5rxhf\" (UID: \"d764ab92-e631-4c4c-a3fc-b71b86a69c33\") " pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"
Nov 04 03:24:38.918572 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:38.918418    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d764ab92-e631-4c4c-a3fc-b71b86a69c33-manager-config\") pod \"jobset-controller-manager-696bfff7cc-5rxhf\" (UID: \"d764ab92-e631-4c4c-a3fc-b71b86a69c33\") " pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"
Nov 04 03:24:38.918572 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:38.918509    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg8kz\" (UniqueName: \"kubernetes.io/projected/d764ab92-e631-4c4c-a3fc-b71b86a69c33-kube-api-access-bg8kz\") pod \"jobset-controller-manager-696bfff7cc-5rxhf\" (UID: \"d764ab92-e631-4c4c-a3fc-b71b86a69c33\") " pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"
Nov 04 03:24:38.918680 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:38.918587    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d764ab92-e631-4c4c-a3fc-b71b86a69c33-metrics-certs\") pod \"jobset-controller-manager-696bfff7cc-5rxhf\" (UID: \"d764ab92-e631-4c4c-a3fc-b71b86a69c33\") " pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"
Nov 04 03:24:38.943752 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:38.943719    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"]
Nov 04 03:24:38.995944 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:38.995913    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Nov 04 03:24:38.995944 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:38.995928    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Nov 04 03:24:38.995944 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:38.995911    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:24:38.996186 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:38.995936    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Nov 04 03:24:38.996186 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:38.995937    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-z4pz7\""
Nov 04 03:24:38.996186 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:38.995913    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 04 03:24:39.019682 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:39.019649    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d764ab92-e631-4c4c-a3fc-b71b86a69c33-cert\") pod \"jobset-controller-manager-696bfff7cc-5rxhf\" (UID: \"d764ab92-e631-4c4c-a3fc-b71b86a69c33\") " pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"
Nov 04 03:24:39.019869 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:39.019694    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d764ab92-e631-4c4c-a3fc-b71b86a69c33-manager-config\") pod \"jobset-controller-manager-696bfff7cc-5rxhf\" (UID: \"d764ab92-e631-4c4c-a3fc-b71b86a69c33\") " pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"
Nov 04 03:24:39.019869 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:39.019724    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg8kz\" (UniqueName: \"kubernetes.io/projected/d764ab92-e631-4c4c-a3fc-b71b86a69c33-kube-api-access-bg8kz\") pod \"jobset-controller-manager-696bfff7cc-5rxhf\" (UID: \"d764ab92-e631-4c4c-a3fc-b71b86a69c33\") " pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"
Nov 04 03:24:39.019869 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:39.019781    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d764ab92-e631-4c4c-a3fc-b71b86a69c33-metrics-certs\") pod \"jobset-controller-manager-696bfff7cc-5rxhf\" (UID: \"d764ab92-e631-4c4c-a3fc-b71b86a69c33\") " pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"
Nov 04 03:24:39.020627 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:39.020604    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d764ab92-e631-4c4c-a3fc-b71b86a69c33-manager-config\") pod \"jobset-controller-manager-696bfff7cc-5rxhf\" (UID: \"d764ab92-e631-4c4c-a3fc-b71b86a69c33\") " pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"
Nov 04 03:24:39.022278 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:39.022256    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d764ab92-e631-4c4c-a3fc-b71b86a69c33-cert\") pod \"jobset-controller-manager-696bfff7cc-5rxhf\" (UID: \"d764ab92-e631-4c4c-a3fc-b71b86a69c33\") " pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"
Nov 04 03:24:39.022454 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:39.022432    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d764ab92-e631-4c4c-a3fc-b71b86a69c33-metrics-certs\") pod \"jobset-controller-manager-696bfff7cc-5rxhf\" (UID: \"d764ab92-e631-4c4c-a3fc-b71b86a69c33\") " pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"
Nov 04 03:24:39.082023 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:39.081994    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg8kz\" (UniqueName: \"kubernetes.io/projected/d764ab92-e631-4c4c-a3fc-b71b86a69c33-kube-api-access-bg8kz\") pod \"jobset-controller-manager-696bfff7cc-5rxhf\" (UID: \"d764ab92-e631-4c4c-a3fc-b71b86a69c33\") " pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"
Nov 04 03:24:39.209062 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:39.208973    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"
Nov 04 03:24:39.591400 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:39.591371    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"]
Nov 04 03:24:39.593860 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:24:39.593820    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd764ab92_e631_4c4c_a3fc_b71b86a69c33.slice/crio-d2cd8629e188a463320b347dac5068ebcc0ee5e0babd6a30c6ce32cd997d7880 WatchSource:0}: Error finding container d2cd8629e188a463320b347dac5068ebcc0ee5e0babd6a30c6ce32cd997d7880: Status 404 returned error can't find the container with id d2cd8629e188a463320b347dac5068ebcc0ee5e0babd6a30c6ce32cd997d7880
Nov 04 03:24:39.646091 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:39.646054    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7659c7d778-65h9l" event={"ID":"da5d1d62-7e67-423b-95ac-8c809a64959c","Type":"ContainerStarted","Data":"5f6c8d005561dcfd89128d668059e48574179d32ab9446dbb47ebfe905b255f4"}
Nov 04 03:24:39.647273 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:39.647250    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf" event={"ID":"d764ab92-e631-4c4c-a3fc-b71b86a69c33","Type":"ContainerStarted","Data":"d2cd8629e188a463320b347dac5068ebcc0ee5e0babd6a30c6ce32cd997d7880"}
Nov 04 03:24:39.763097 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:39.763040    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7659c7d778-65h9l" podStartSLOduration=1.363922723 podStartE2EDuration="4.763025044s" totalImagesPullingTime="3.399102321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:24:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:24:39.762443789 +0000 UTC m=+445.967044108" watchObservedRunningTime="2026-11-04 03:24:39.763025044 +0000 UTC m=+445.967625345"
Nov 04 03:24:44.670492 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:44.670453    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf" event={"ID":"d764ab92-e631-4c4c-a3fc-b71b86a69c33","Type":"ContainerStarted","Data":"21be76ab236bc7a3d49bcc2346933fd5abec938f91a15a0d63afe0e0f2392859"}
Nov 04 03:24:44.670934 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:44.670573    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"
Nov 04 03:24:44.790253 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:44.790200    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf" podStartSLOduration=2.497706375 podStartE2EDuration="6.790185928s" totalImagesPullingTime="4.292479553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:24:44.78866724 +0000 UTC m=+450.993267541" watchObservedRunningTime="2026-11-04 03:24:44.790185928 +0000 UTC m=+450.994786230"
Nov 04 03:24:55.681187 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:24:55.681148    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-696bfff7cc-5rxhf"
Nov 04 03:24:58.729749 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:24:58.729715    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:25:28.727803 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:25:28.727726    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:25:58.729630 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:25:58.729588    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:26:09.690991 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:09.690956    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-n2wm8"]
Nov 04 03:26:09.693625 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:09.693606    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-n2wm8"
Nov 04 03:26:09.797007 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:09.796959    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 04 03:26:09.797007 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:09.796987    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 04 03:26:09.797007 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:09.796964    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m42fv\" (UniqueName: \"kubernetes.io/projected/9754c3f0-c1b7-431d-844e-0536a6654972-kube-api-access-m42fv\") pod \"odh-model-controller-769875db56-n2wm8\" (UID: \"9754c3f0-c1b7-431d-844e-0536a6654972\") " pod="opendatahub/odh-model-controller-769875db56-n2wm8"
Nov 04 03:26:09.797007 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:09.797012    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Nov 04 03:26:09.797351 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:09.797085    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9754c3f0-c1b7-431d-844e-0536a6654972-cert\") pod \"odh-model-controller-769875db56-n2wm8\" (UID: \"9754c3f0-c1b7-431d-844e-0536a6654972\") " pod="opendatahub/odh-model-controller-769875db56-n2wm8"
Nov 04 03:26:09.797351 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:09.797104    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9754c3f0-c1b7-431d-844e-0536a6654972-metrics-certs\") pod \"odh-model-controller-769875db56-n2wm8\" (UID: \"9754c3f0-c1b7-431d-844e-0536a6654972\") " pod="opendatahub/odh-model-controller-769875db56-n2wm8"
Nov 04 03:26:09.797351 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:09.797157    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-mrgzr\""
Nov 04 03:26:09.797351 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:09.797215    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Nov 04 03:26:09.798503 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:09.798478    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-n2wm8"]
Nov 04 03:26:09.897900 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:09.897868    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9754c3f0-c1b7-431d-844e-0536a6654972-cert\") pod \"odh-model-controller-769875db56-n2wm8\" (UID: \"9754c3f0-c1b7-431d-844e-0536a6654972\") " pod="opendatahub/odh-model-controller-769875db56-n2wm8"
Nov 04 03:26:09.898130 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:09.897907    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9754c3f0-c1b7-431d-844e-0536a6654972-metrics-certs\") pod \"odh-model-controller-769875db56-n2wm8\" (UID: \"9754c3f0-c1b7-431d-844e-0536a6654972\") " pod="opendatahub/odh-model-controller-769875db56-n2wm8"
Nov 04 03:26:09.898130 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:09.897960    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m42fv\" (UniqueName: \"kubernetes.io/projected/9754c3f0-c1b7-431d-844e-0536a6654972-kube-api-access-m42fv\") pod \"odh-model-controller-769875db56-n2wm8\" (UID: \"9754c3f0-c1b7-431d-844e-0536a6654972\") " pod="opendatahub/odh-model-controller-769875db56-n2wm8"
Nov 04 03:26:09.900291 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:09.900268    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9754c3f0-c1b7-431d-844e-0536a6654972-metrics-certs\") pod \"odh-model-controller-769875db56-n2wm8\" (UID: \"9754c3f0-c1b7-431d-844e-0536a6654972\") " pod="opendatahub/odh-model-controller-769875db56-n2wm8"
Nov 04 03:26:09.900393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:09.900318    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9754c3f0-c1b7-431d-844e-0536a6654972-cert\") pod \"odh-model-controller-769875db56-n2wm8\" (UID: \"9754c3f0-c1b7-431d-844e-0536a6654972\") " pod="opendatahub/odh-model-controller-769875db56-n2wm8"
Nov 04 03:26:09.959753 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:09.959683    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m42fv\" (UniqueName: \"kubernetes.io/projected/9754c3f0-c1b7-431d-844e-0536a6654972-kube-api-access-m42fv\") pod \"odh-model-controller-769875db56-n2wm8\" (UID: \"9754c3f0-c1b7-431d-844e-0536a6654972\") " pod="opendatahub/odh-model-controller-769875db56-n2wm8"
Nov 04 03:26:10.005890 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:10.005852    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-n2wm8"
Nov 04 03:26:10.378983 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:10.378944    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-n2wm8"]
Nov 04 03:26:10.381915 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:26:10.381884    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9754c3f0_c1b7_431d_844e_0536a6654972.slice/crio-6eab60275254204371db33fa7e625d139e26ba095fc57882c8ab5e0c74c302f2 WatchSource:0}: Error finding container 6eab60275254204371db33fa7e625d139e26ba095fc57882c8ab5e0c74c302f2: Status 404 returned error can't find the container with id 6eab60275254204371db33fa7e625d139e26ba095fc57882c8ab5e0c74c302f2
Nov 04 03:26:10.729620 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:10.729539    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"]
Nov 04 03:26:10.730030 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:10.729777    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml" podUID="a5a024f1-d038-44c2-bf29-6d3365192f7f" containerName="discovery" containerID="cri-o://9c705c8599a2a50718f43f6d5ad31b4595c883bbe469ee339343acc1a8c3928f" gracePeriod=30
Nov 04 03:26:11.017361 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.017330    4521 generic.go:356] "Generic (PLEG): container finished" podID="a5a024f1-d038-44c2-bf29-6d3365192f7f" containerID="9c705c8599a2a50718f43f6d5ad31b4595c883bbe469ee339343acc1a8c3928f" exitCode=0
Nov 04 03:26:11.017548 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.017405    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml" event={"ID":"a5a024f1-d038-44c2-bf29-6d3365192f7f","Type":"ContainerDied","Data":"9c705c8599a2a50718f43f6d5ad31b4595c883bbe469ee339343acc1a8c3928f"}
Nov 04 03:26:11.018960 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.018930    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-n2wm8" event={"ID":"9754c3f0-c1b7-431d-844e-0536a6654972","Type":"ContainerStarted","Data":"6eab60275254204371db33fa7e625d139e26ba095fc57882c8ab5e0c74c302f2"}
Nov 04 03:26:11.068118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.068094    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:26:11.211000 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.210965    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-kubeconfig\") pod \"a5a024f1-d038-44c2-bf29-6d3365192f7f\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") "
Nov 04 03:26:11.211155 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.211024    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-csr-ca-configmap\") pod \"a5a024f1-d038-44c2-bf29-6d3365192f7f\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") "
Nov 04 03:26:11.211155 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.211050    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5a024f1-d038-44c2-bf29-6d3365192f7f-kube-api-access-2vnxm\" (UniqueName: \"kubernetes.io/projected/a5a024f1-d038-44c2-bf29-6d3365192f7f-kube-api-access-2vnxm\") pod \"a5a024f1-d038-44c2-bf29-6d3365192f7f\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") "
Nov 04 03:26:11.211248 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.211160    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-cacerts\" (UniqueName: \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-cacerts\") pod \"a5a024f1-d038-44c2-bf29-6d3365192f7f\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") "
Nov 04 03:26:11.211248 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.211206    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5a024f1-d038-44c2-bf29-6d3365192f7f-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5a024f1-d038-44c2-bf29-6d3365192f7f-local-certs\") pod \"a5a024f1-d038-44c2-bf29-6d3365192f7f\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") "
Nov 04 03:26:11.211364 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.211255    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-csr-dns-cert\") pod \"a5a024f1-d038-44c2-bf29-6d3365192f7f\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") "
Nov 04 03:26:11.211364 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.211328    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-token\" (UniqueName: \"kubernetes.io/projected/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-token\") pod \"a5a024f1-d038-44c2-bf29-6d3365192f7f\" (UID: \"a5a024f1-d038-44c2-bf29-6d3365192f7f\") "
Nov 04 03:26:11.211475 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.211441    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-csr-ca-configmap" pod "a5a024f1-d038-44c2-bf29-6d3365192f7f" (UID: "a5a024f1-d038-44c2-bf29-6d3365192f7f"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:26:11.211665 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.211640    4521 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-csr-ca-configmap\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:26:11.213667 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.213638    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-csr-dns-cert" pod "a5a024f1-d038-44c2-bf29-6d3365192f7f" (UID: "a5a024f1-d038-44c2-bf29-6d3365192f7f"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:26:11.213785 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.213641    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5a024f1-d038-44c2-bf29-6d3365192f7f-kube-api-access-2vnxm" pod "a5a024f1-d038-44c2-bf29-6d3365192f7f" (UID: "a5a024f1-d038-44c2-bf29-6d3365192f7f"). InnerVolumeSpecName "kube-api-access-2vnxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:26:11.213785 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.213699    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-kubeconfig" pod "a5a024f1-d038-44c2-bf29-6d3365192f7f" (UID: "a5a024f1-d038-44c2-bf29-6d3365192f7f"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:26:11.213947 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.213922    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-cacerts" pod "a5a024f1-d038-44c2-bf29-6d3365192f7f" (UID: "a5a024f1-d038-44c2-bf29-6d3365192f7f"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:26:11.214057 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.214035    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-token" pod "a5a024f1-d038-44c2-bf29-6d3365192f7f" (UID: "a5a024f1-d038-44c2-bf29-6d3365192f7f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:26:11.214120 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.214035    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5a024f1-d038-44c2-bf29-6d3365192f7f-local-certs" pod "a5a024f1-d038-44c2-bf29-6d3365192f7f" (UID: "a5a024f1-d038-44c2-bf29-6d3365192f7f"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:26:11.313050 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.313018    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vnxm\" (UniqueName: \"kubernetes.io/projected/a5a024f1-d038-44c2-bf29-6d3365192f7f-kube-api-access-2vnxm\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:26:11.313050 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.313047    4521 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-cacerts\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:26:11.313245 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.313058    4521 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5a024f1-d038-44c2-bf29-6d3365192f7f-local-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:26:11.313245 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.313068    4521 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-csr-dns-cert\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:26:11.313245 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.313077    4521 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-token\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:26:11.313245 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:11.313085    4521 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a5a024f1-d038-44c2-bf29-6d3365192f7f-istio-kubeconfig\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:26:12.030934 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:12.030882    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml" event={"ID":"a5a024f1-d038-44c2-bf29-6d3365192f7f","Type":"ContainerDied","Data":"fb1433f0b9f54785e697a6a4bf23412fef9743103d91034e5a8e8004771f133e"}
Nov 04 03:26:12.030934 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:12.030919    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"
Nov 04 03:26:12.031463 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:12.030995    4521 scope.go:122] "RemoveContainer" containerID="9c705c8599a2a50718f43f6d5ad31b4595c883bbe469ee339343acc1a8c3928f"
Nov 04 03:26:12.248045 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:12.247960    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"]
Nov 04 03:26:12.298716 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:12.298678    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-mzwml"]
Nov 04 03:26:13.910520 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:13.910433    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5a024f1-d038-44c2-bf29-6d3365192f7f" path="/var/lib/kubelet/pods/a5a024f1-d038-44c2-bf29-6d3365192f7f/volumes"
Nov 04 03:26:14.042900 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:14.042869    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-n2wm8" event={"ID":"9754c3f0-c1b7-431d-844e-0536a6654972","Type":"ContainerStarted","Data":"50f0b8b7b1f97d15798d0afa4460a07662e967cac4f3cc93e0781854c604f41a"}
Nov 04 03:26:14.043059 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:14.043011    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-n2wm8"
Nov 04 03:26:14.160769 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:14.160656    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-n2wm8" podStartSLOduration=1.976249704 podStartE2EDuration="5.160640125s" totalImagesPullingTime="3.184390421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:26:14.159682462 +0000 UTC m=+540.364282763" watchObservedRunningTime="2026-11-04 03:26:14.160640125 +0000 UTC m=+540.365240426"
Nov 04 03:26:18.094698 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.094664    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"]
Nov 04 03:26:18.095218 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.094998    4521 cpu_manager.go:414] "containerMap: removing container" podUID="a5a024f1-d038-44c2-bf29-6d3365192f7f" containerName="discovery"
Nov 04 03:26:18.095218 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.095009    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5a024f1-d038-44c2-bf29-6d3365192f7f" containerName="discovery"
Nov 04 03:26:18.095218 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.095091    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5a024f1-d038-44c2-bf29-6d3365192f7f" containerName="discovery"
Nov 04 03:26:18.098084 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.098069    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:18.168542 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.168510    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d26e08a-cc00-44a6-8749-dab26c4e787a-maas-api-tls\") pod \"maas-api-665d94ccb6-6hj8k\" (UID: \"4d26e08a-cc00-44a6-8749-dab26c4e787a\") " pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:18.168696 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.168570    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sh6z\" (UniqueName: \"kubernetes.io/projected/4d26e08a-cc00-44a6-8749-dab26c4e787a-kube-api-access-5sh6z\") pod \"maas-api-665d94ccb6-6hj8k\" (UID: \"4d26e08a-cc00-44a6-8749-dab26c4e787a\") " pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:18.168696 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.168646    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d26e08a-cc00-44a6-8749-dab26c4e787a-openshift-service-ca\") pod \"maas-api-665d94ccb6-6hj8k\" (UID: \"4d26e08a-cc00-44a6-8749-dab26c4e787a\") " pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:18.168696 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.168683    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d26e08a-cc00-44a6-8749-dab26c4e787a-metrics-certs\") pod \"maas-api-665d94ccb6-6hj8k\" (UID: \"4d26e08a-cc00-44a6-8749-dab26c4e787a\") " pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:18.201550 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.201517    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 03:26:18.201727 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.201608    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 03:26:18.201727 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.201641    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 04 03:26:18.201727 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.201696    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 04 03:26:18.201928 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.201786    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-cpkkm\""
Nov 04 03:26:18.203120 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.203098    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"]
Nov 04 03:26:18.269425 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.269388    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sh6z\" (UniqueName: \"kubernetes.io/projected/4d26e08a-cc00-44a6-8749-dab26c4e787a-kube-api-access-5sh6z\") pod \"maas-api-665d94ccb6-6hj8k\" (UID: \"4d26e08a-cc00-44a6-8749-dab26c4e787a\") " pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:18.269605 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.269434    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d26e08a-cc00-44a6-8749-dab26c4e787a-openshift-service-ca\") pod \"maas-api-665d94ccb6-6hj8k\" (UID: \"4d26e08a-cc00-44a6-8749-dab26c4e787a\") " pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:18.269605 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.269468    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d26e08a-cc00-44a6-8749-dab26c4e787a-metrics-certs\") pod \"maas-api-665d94ccb6-6hj8k\" (UID: \"4d26e08a-cc00-44a6-8749-dab26c4e787a\") " pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:18.269605 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.269493    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d26e08a-cc00-44a6-8749-dab26c4e787a-maas-api-tls\") pod \"maas-api-665d94ccb6-6hj8k\" (UID: \"4d26e08a-cc00-44a6-8749-dab26c4e787a\") " pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:18.270248 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.270225    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d26e08a-cc00-44a6-8749-dab26c4e787a-openshift-service-ca\") pod \"maas-api-665d94ccb6-6hj8k\" (UID: \"4d26e08a-cc00-44a6-8749-dab26c4e787a\") " pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:18.271994 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.271973    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d26e08a-cc00-44a6-8749-dab26c4e787a-metrics-certs\") pod \"maas-api-665d94ccb6-6hj8k\" (UID: \"4d26e08a-cc00-44a6-8749-dab26c4e787a\") " pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:18.272127 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.272112    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d26e08a-cc00-44a6-8749-dab26c4e787a-maas-api-tls\") pod \"maas-api-665d94ccb6-6hj8k\" (UID: \"4d26e08a-cc00-44a6-8749-dab26c4e787a\") " pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:18.331426 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.331388    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sh6z\" (UniqueName: \"kubernetes.io/projected/4d26e08a-cc00-44a6-8749-dab26c4e787a-kube-api-access-5sh6z\") pod \"maas-api-665d94ccb6-6hj8k\" (UID: \"4d26e08a-cc00-44a6-8749-dab26c4e787a\") " pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:18.410520 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.410443    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:18.780873 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:18.780827    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"]
Nov 04 03:26:18.784096 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:26:18.784071    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d26e08a_cc00_44a6_8749_dab26c4e787a.slice/crio-ce022cbbc56bef0bbebb2b468c794dcb383007bccb1baf72ae3a51fb14c65d6b WatchSource:0}: Error finding container ce022cbbc56bef0bbebb2b468c794dcb383007bccb1baf72ae3a51fb14c65d6b: Status 404 returned error can't find the container with id ce022cbbc56bef0bbebb2b468c794dcb383007bccb1baf72ae3a51fb14c65d6b
Nov 04 03:26:19.063971 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:19.063869    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k" event={"ID":"4d26e08a-cc00-44a6-8749-dab26c4e787a","Type":"ContainerStarted","Data":"ce022cbbc56bef0bbebb2b468c794dcb383007bccb1baf72ae3a51fb14c65d6b"}
Nov 04 03:26:25.049578 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:25.049549    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-n2wm8"
Nov 04 03:26:25.091306 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:25.091277    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k" event={"ID":"4d26e08a-cc00-44a6-8749-dab26c4e787a","Type":"ContainerStarted","Data":"34d9ae41b7ecbb9134fe8cbc31a39d3822784ae672dda8ddeef0cf3742cafc9d"}
Nov 04 03:26:25.091453 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:25.091432    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:25.287216 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:25.287157    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k" podStartSLOduration=1.065551127 podStartE2EDuration="7.287141158s" totalImagesPullingTime="6.221590031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:26:25.285372624 +0000 UTC m=+551.489972928" watchObservedRunningTime="2026-11-04 03:26:25.287141158 +0000 UTC m=+551.491741460"
Nov 04 03:26:28.729522 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:26:28.729486    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:26:31.100743 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:31.100712    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:32.170441 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:32.170399    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"]
Nov 04 03:26:32.173808 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:32.173789    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"
Nov 04 03:26:32.278097 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:32.278067    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"]
Nov 04 03:26:32.279165 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:32.279145    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccda3357-d0c9-467f-b765-7cb9d9d5a532-maas-api-tls\") pod \"maas-api-cbd8487c-8dvdn\" (UID: \"ccda3357-d0c9-467f-b765-7cb9d9d5a532\") " pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"
Nov 04 03:26:32.279234 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:32.279186    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccda3357-d0c9-467f-b765-7cb9d9d5a532-openshift-service-ca\") pod \"maas-api-cbd8487c-8dvdn\" (UID: \"ccda3357-d0c9-467f-b765-7cb9d9d5a532\") " pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"
Nov 04 03:26:32.279234 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:32.279204    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvb2l\" (UniqueName: \"kubernetes.io/projected/ccda3357-d0c9-467f-b765-7cb9d9d5a532-kube-api-access-pvb2l\") pod \"maas-api-cbd8487c-8dvdn\" (UID: \"ccda3357-d0c9-467f-b765-7cb9d9d5a532\") " pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"
Nov 04 03:26:32.279336 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:32.279317    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccda3357-d0c9-467f-b765-7cb9d9d5a532-metrics-certs\") pod \"maas-api-cbd8487c-8dvdn\" (UID: \"ccda3357-d0c9-467f-b765-7cb9d9d5a532\") " pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"
Nov 04 03:26:32.380108 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:32.380068    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccda3357-d0c9-467f-b765-7cb9d9d5a532-maas-api-tls\") pod \"maas-api-cbd8487c-8dvdn\" (UID: \"ccda3357-d0c9-467f-b765-7cb9d9d5a532\") " pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"
Nov 04 03:26:32.380309 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:32.380132    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccda3357-d0c9-467f-b765-7cb9d9d5a532-openshift-service-ca\") pod \"maas-api-cbd8487c-8dvdn\" (UID: \"ccda3357-d0c9-467f-b765-7cb9d9d5a532\") " pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"
Nov 04 03:26:32.380309 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:32.380158    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvb2l\" (UniqueName: \"kubernetes.io/projected/ccda3357-d0c9-467f-b765-7cb9d9d5a532-kube-api-access-pvb2l\") pod \"maas-api-cbd8487c-8dvdn\" (UID: \"ccda3357-d0c9-467f-b765-7cb9d9d5a532\") " pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"
Nov 04 03:26:32.380309 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:32.380200    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccda3357-d0c9-467f-b765-7cb9d9d5a532-metrics-certs\") pod \"maas-api-cbd8487c-8dvdn\" (UID: \"ccda3357-d0c9-467f-b765-7cb9d9d5a532\") " pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"
Nov 04 03:26:32.380753 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:32.380730    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccda3357-d0c9-467f-b765-7cb9d9d5a532-openshift-service-ca\") pod \"maas-api-cbd8487c-8dvdn\" (UID: \"ccda3357-d0c9-467f-b765-7cb9d9d5a532\") " pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"
Nov 04 03:26:32.382762 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:32.382736    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccda3357-d0c9-467f-b765-7cb9d9d5a532-metrics-certs\") pod \"maas-api-cbd8487c-8dvdn\" (UID: \"ccda3357-d0c9-467f-b765-7cb9d9d5a532\") " pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"
Nov 04 03:26:32.382867 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:32.382736    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccda3357-d0c9-467f-b765-7cb9d9d5a532-maas-api-tls\") pod \"maas-api-cbd8487c-8dvdn\" (UID: \"ccda3357-d0c9-467f-b765-7cb9d9d5a532\") " pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"
Nov 04 03:26:32.441962 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:32.441896    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvb2l\" (UniqueName: \"kubernetes.io/projected/ccda3357-d0c9-467f-b765-7cb9d9d5a532-kube-api-access-pvb2l\") pod \"maas-api-cbd8487c-8dvdn\" (UID: \"ccda3357-d0c9-467f-b765-7cb9d9d5a532\") " pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"
Nov 04 03:26:32.487003 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:32.486971    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"
Nov 04 03:26:32.862903 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:32.862875    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"]
Nov 04 03:26:32.865524 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:26:32.865488    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccda3357_d0c9_467f_b765_7cb9d9d5a532.slice/crio-42a404829c929bf73e2ef576f3ac34d89fc9f5b8f22515194b1b08c20768264f WatchSource:0}: Error finding container 42a404829c929bf73e2ef576f3ac34d89fc9f5b8f22515194b1b08c20768264f: Status 404 returned error can't find the container with id 42a404829c929bf73e2ef576f3ac34d89fc9f5b8f22515194b1b08c20768264f
Nov 04 03:26:33.125098 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:33.125008    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn" event={"ID":"ccda3357-d0c9-467f-b765-7cb9d9d5a532","Type":"ContainerStarted","Data":"42a404829c929bf73e2ef576f3ac34d89fc9f5b8f22515194b1b08c20768264f"}
Nov 04 03:26:34.130323 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:34.130287    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn" event={"ID":"ccda3357-d0c9-467f-b765-7cb9d9d5a532","Type":"ContainerStarted","Data":"a8d3617b35561620828c59d51a8669da3c76007208f32797cb9b91ff994d9d3d"}
Nov 04 03:26:34.130740 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:34.130374    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"
Nov 04 03:26:34.248935 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:34.248749    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn" podStartSLOduration=1.366413794 podStartE2EDuration="2.24873078s" totalImagesPullingTime="882.316986ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:26:34.248357354 +0000 UTC m=+560.452957678" watchObservedRunningTime="2026-11-04 03:26:34.24873078 +0000 UTC m=+560.453331081"
Nov 04 03:26:40.139818 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:40.139789    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-cbd8487c-8dvdn"
Nov 04 03:26:40.310520 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:40.310476    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"]
Nov 04 03:26:40.310811 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:40.310759    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k" podUID="4d26e08a-cc00-44a6-8749-dab26c4e787a" containerName="maas-api" containerID="cri-o://34d9ae41b7ecbb9134fe8cbc31a39d3822784ae672dda8ddeef0cf3742cafc9d" gracePeriod=30
Nov 04 03:26:40.558214 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:40.558189    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:40.652692 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:40.652658    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4d26e08a-cc00-44a6-8749-dab26c4e787a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d26e08a-cc00-44a6-8749-dab26c4e787a-openshift-service-ca\") pod \"4d26e08a-cc00-44a6-8749-dab26c4e787a\" (UID: \"4d26e08a-cc00-44a6-8749-dab26c4e787a\") "
Nov 04 03:26:40.652692 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:40.652697    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4d26e08a-cc00-44a6-8749-dab26c4e787a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d26e08a-cc00-44a6-8749-dab26c4e787a-metrics-certs\") pod \"4d26e08a-cc00-44a6-8749-dab26c4e787a\" (UID: \"4d26e08a-cc00-44a6-8749-dab26c4e787a\") "
Nov 04 03:26:40.652958 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:40.652748    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4d26e08a-cc00-44a6-8749-dab26c4e787a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d26e08a-cc00-44a6-8749-dab26c4e787a-maas-api-tls\") pod \"4d26e08a-cc00-44a6-8749-dab26c4e787a\" (UID: \"4d26e08a-cc00-44a6-8749-dab26c4e787a\") "
Nov 04 03:26:40.652958 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:40.652877    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d26e08a-cc00-44a6-8749-dab26c4e787a-kube-api-access-5sh6z\" (UniqueName: \"kubernetes.io/projected/4d26e08a-cc00-44a6-8749-dab26c4e787a-kube-api-access-5sh6z\") pod \"4d26e08a-cc00-44a6-8749-dab26c4e787a\" (UID: \"4d26e08a-cc00-44a6-8749-dab26c4e787a\") "
Nov 04 03:26:40.653174 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:40.653143    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d26e08a-cc00-44a6-8749-dab26c4e787a-openshift-service-ca" pod "4d26e08a-cc00-44a6-8749-dab26c4e787a" (UID: "4d26e08a-cc00-44a6-8749-dab26c4e787a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:26:40.655139 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:40.655114    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d26e08a-cc00-44a6-8749-dab26c4e787a-maas-api-tls" pod "4d26e08a-cc00-44a6-8749-dab26c4e787a" (UID: "4d26e08a-cc00-44a6-8749-dab26c4e787a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:26:40.655245 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:40.655160    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d26e08a-cc00-44a6-8749-dab26c4e787a-kube-api-access-5sh6z" pod "4d26e08a-cc00-44a6-8749-dab26c4e787a" (UID: "4d26e08a-cc00-44a6-8749-dab26c4e787a"). InnerVolumeSpecName "kube-api-access-5sh6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:26:40.655245 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:40.655174    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d26e08a-cc00-44a6-8749-dab26c4e787a-metrics-certs" pod "4d26e08a-cc00-44a6-8749-dab26c4e787a" (UID: "4d26e08a-cc00-44a6-8749-dab26c4e787a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:26:40.754319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:40.754230    4521 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d26e08a-cc00-44a6-8749-dab26c4e787a-openshift-service-ca\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:26:40.754319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:40.754261    4521 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d26e08a-cc00-44a6-8749-dab26c4e787a-metrics-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:26:40.754319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:40.754273    4521 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d26e08a-cc00-44a6-8749-dab26c4e787a-maas-api-tls\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:26:40.754319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:40.754282    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sh6z\" (UniqueName: \"kubernetes.io/projected/4d26e08a-cc00-44a6-8749-dab26c4e787a-kube-api-access-5sh6z\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:26:41.158715 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:41.158683    4521 generic.go:356] "Generic (PLEG): container finished" podID="4d26e08a-cc00-44a6-8749-dab26c4e787a" containerID="34d9ae41b7ecbb9134fe8cbc31a39d3822784ae672dda8ddeef0cf3742cafc9d" exitCode=0
Nov 04 03:26:41.159204 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:41.158736    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"
Nov 04 03:26:41.159204 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:41.158754    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k" event={"ID":"4d26e08a-cc00-44a6-8749-dab26c4e787a","Type":"ContainerDied","Data":"34d9ae41b7ecbb9134fe8cbc31a39d3822784ae672dda8ddeef0cf3742cafc9d"}
Nov 04 03:26:41.159204 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:41.158798    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k" event={"ID":"4d26e08a-cc00-44a6-8749-dab26c4e787a","Type":"ContainerDied","Data":"ce022cbbc56bef0bbebb2b468c794dcb383007bccb1baf72ae3a51fb14c65d6b"}
Nov 04 03:26:41.159204 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:41.158820    4521 scope.go:122] "RemoveContainer" containerID="34d9ae41b7ecbb9134fe8cbc31a39d3822784ae672dda8ddeef0cf3742cafc9d"
Nov 04 03:26:41.167663 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:41.167640    4521 scope.go:122] "RemoveContainer" containerID="34d9ae41b7ecbb9134fe8cbc31a39d3822784ae672dda8ddeef0cf3742cafc9d"
Nov 04 03:26:41.167952 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:26:41.167928    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34d9ae41b7ecbb9134fe8cbc31a39d3822784ae672dda8ddeef0cf3742cafc9d\": container with ID starting with 34d9ae41b7ecbb9134fe8cbc31a39d3822784ae672dda8ddeef0cf3742cafc9d not found: ID does not exist" containerID="34d9ae41b7ecbb9134fe8cbc31a39d3822784ae672dda8ddeef0cf3742cafc9d"
Nov 04 03:26:41.168010 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:41.167963    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34d9ae41b7ecbb9134fe8cbc31a39d3822784ae672dda8ddeef0cf3742cafc9d"} err="failed to get container status \"34d9ae41b7ecbb9134fe8cbc31a39d3822784ae672dda8ddeef0cf3742cafc9d\": rpc error: code = NotFound desc = could not find container \"34d9ae41b7ecbb9134fe8cbc31a39d3822784ae672dda8ddeef0cf3742cafc9d\": container with ID starting with 34d9ae41b7ecbb9134fe8cbc31a39d3822784ae672dda8ddeef0cf3742cafc9d not found: ID does not exist"
Nov 04 03:26:41.324064 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:41.324029    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"]
Nov 04 03:26:41.328006 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:41.327979    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-665d94ccb6-6hj8k"]
Nov 04 03:26:41.909489 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:26:41.909454    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d26e08a-cc00-44a6-8749-dab26c4e787a" path="/var/lib/kubelet/pods/4d26e08a-cc00-44a6-8749-dab26c4e787a/volumes"
Nov 04 03:26:58.728677 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:26:58.728570    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:27:28.729228 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:27:28.729192    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:27:51.996376 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:51.996338    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"]
Nov 04 03:27:51.997008 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:51.996730    4521 cpu_manager.go:414] "containerMap: removing container" podUID="4d26e08a-cc00-44a6-8749-dab26c4e787a" containerName="maas-api"
Nov 04 03:27:51.997008 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:51.996748    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d26e08a-cc00-44a6-8749-dab26c4e787a" containerName="maas-api"
Nov 04 03:27:51.997008 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:51.996827    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d26e08a-cc00-44a6-8749-dab26c4e787a" containerName="maas-api"
Nov 04 03:27:52.000173 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:52.000153    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"
Nov 04 03:27:52.103951 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:52.103914    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Nov 04 03:27:52.103951 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:52.103920    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-r2mzp\""
Nov 04 03:27:52.103951 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:52.103958    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Nov 04 03:27:52.105106 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:52.105085    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"]
Nov 04 03:27:52.180991 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:52.180955    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a12d5e79-b031-4add-8f08-27e6a4866876-metrics-certs\") pod \"ai-gateway-controller-5d8dddb676-jcvfc\" (UID: \"a12d5e79-b031-4add-8f08-27e6a4866876\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"
Nov 04 03:27:52.180991 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:52.180996    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9spc\" (UniqueName: \"kubernetes.io/projected/a12d5e79-b031-4add-8f08-27e6a4866876-kube-api-access-v9spc\") pod \"ai-gateway-controller-5d8dddb676-jcvfc\" (UID: \"a12d5e79-b031-4add-8f08-27e6a4866876\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"
Nov 04 03:27:52.282210 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:52.282185    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a12d5e79-b031-4add-8f08-27e6a4866876-metrics-certs\") pod \"ai-gateway-controller-5d8dddb676-jcvfc\" (UID: \"a12d5e79-b031-4add-8f08-27e6a4866876\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"
Nov 04 03:27:52.282363 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:52.282223    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9spc\" (UniqueName: \"kubernetes.io/projected/a12d5e79-b031-4add-8f08-27e6a4866876-kube-api-access-v9spc\") pod \"ai-gateway-controller-5d8dddb676-jcvfc\" (UID: \"a12d5e79-b031-4add-8f08-27e6a4866876\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"
Nov 04 03:27:52.284633 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:52.284608    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a12d5e79-b031-4add-8f08-27e6a4866876-metrics-certs\") pod \"ai-gateway-controller-5d8dddb676-jcvfc\" (UID: \"a12d5e79-b031-4add-8f08-27e6a4866876\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"
Nov 04 03:27:52.351730 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:52.351698    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9spc\" (UniqueName: \"kubernetes.io/projected/a12d5e79-b031-4add-8f08-27e6a4866876-kube-api-access-v9spc\") pod \"ai-gateway-controller-5d8dddb676-jcvfc\" (UID: \"a12d5e79-b031-4add-8f08-27e6a4866876\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"
Nov 04 03:27:52.614012 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:52.613929    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"
Nov 04 03:27:52.989094 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:52.988870    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"]
Nov 04 03:27:52.991903 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:27:52.991873    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda12d5e79_b031_4add_8f08_27e6a4866876.slice/crio-816f12b0d4bcc93313acd3928abc67b82a4e6b86bef2595238298634a2cee3e9 WatchSource:0}: Error finding container 816f12b0d4bcc93313acd3928abc67b82a4e6b86bef2595238298634a2cee3e9: Status 404 returned error can't find the container with id 816f12b0d4bcc93313acd3928abc67b82a4e6b86bef2595238298634a2cee3e9
Nov 04 03:27:52.993251 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:52.993231    4521 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:27:53.455015 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:53.454977    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc" event={"ID":"a12d5e79-b031-4add-8f08-27e6a4866876","Type":"ContainerStarted","Data":"816f12b0d4bcc93313acd3928abc67b82a4e6b86bef2595238298634a2cee3e9"}
Nov 04 03:27:55.465169 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:55.465088    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc" event={"ID":"a12d5e79-b031-4add-8f08-27e6a4866876","Type":"ContainerStarted","Data":"cbdb2eb2c30ac78103924aa0a84aee8541b94cc708cab05b0fdfd92b73d764fb"}
Nov 04 03:27:55.465169 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:55.465151    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"
Nov 04 03:27:55.583461 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:55.583401    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc" podStartSLOduration=2.360761137 podStartE2EDuration="4.58338469s" totalImagesPullingTime="2.222623553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:27:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:27:55.582816142 +0000 UTC m=+641.787416468" watchObservedRunningTime="2026-11-04 03:27:55.58338469 +0000 UTC m=+641.787984992"
Nov 04 03:27:58.732598 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:27:58.732557    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:27:59.594854 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.594806    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-chld2"]
Nov 04 03:27:59.598047 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.598029    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-chld2"
Nov 04 03:27:59.649363 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.649333    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d770e48e-5587-4da6-838b-f7b52d18c86a-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-chld2\" (UID: \"d770e48e-5587-4da6-838b-f7b52d18c86a\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-chld2"
Nov 04 03:27:59.649486 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.649376    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58zc4\" (UniqueName: \"kubernetes.io/projected/d770e48e-5587-4da6-838b-f7b52d18c86a-kube-api-access-58zc4\") pod \"payload-pre-processing-7b98bb678c-chld2\" (UID: \"d770e48e-5587-4da6-838b-f7b52d18c86a\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-chld2"
Nov 04 03:27:59.651042 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.651018    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Nov 04 03:27:59.701421 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.701388    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-t4tj9\""
Nov 04 03:27:59.702669 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.702647    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-chld2"]
Nov 04 03:27:59.749908 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.749878    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d770e48e-5587-4da6-838b-f7b52d18c86a-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-chld2\" (UID: \"d770e48e-5587-4da6-838b-f7b52d18c86a\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-chld2"
Nov 04 03:27:59.750274 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.749930    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58zc4\" (UniqueName: \"kubernetes.io/projected/d770e48e-5587-4da6-838b-f7b52d18c86a-kube-api-access-58zc4\") pod \"payload-pre-processing-7b98bb678c-chld2\" (UID: \"d770e48e-5587-4da6-838b-f7b52d18c86a\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-chld2"
Nov 04 03:27:59.750583 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.750563    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d770e48e-5587-4da6-838b-f7b52d18c86a-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-chld2\" (UID: \"d770e48e-5587-4da6-838b-f7b52d18c86a\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-chld2"
Nov 04 03:27:59.752968 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.752937    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-khhp6"]
Nov 04 03:27:59.756280 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.756258    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-khhp6"
Nov 04 03:27:59.787180 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.787155    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-khhp6"]
Nov 04 03:27:59.811887 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.811839    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58zc4\" (UniqueName: \"kubernetes.io/projected/d770e48e-5587-4da6-838b-f7b52d18c86a-kube-api-access-58zc4\") pod \"payload-pre-processing-7b98bb678c-chld2\" (UID: \"d770e48e-5587-4da6-838b-f7b52d18c86a\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-chld2"
Nov 04 03:27:59.851170 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.851096    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5kkg\" (UniqueName: \"kubernetes.io/projected/cabd37d3-8208-4220-80c4-2d28ed781ce3-kube-api-access-d5kkg\") pod \"payload-processing-7bb9489676-khhp6\" (UID: \"cabd37d3-8208-4220-80c4-2d28ed781ce3\") " pod="openshift-ingress/payload-processing-7bb9489676-khhp6"
Nov 04 03:27:59.851170 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.851161    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cabd37d3-8208-4220-80c4-2d28ed781ce3-plugins-config-volume\") pod \"payload-processing-7bb9489676-khhp6\" (UID: \"cabd37d3-8208-4220-80c4-2d28ed781ce3\") " pod="openshift-ingress/payload-processing-7bb9489676-khhp6"
Nov 04 03:27:59.910918 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.910892    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-chld2"
Nov 04 03:27:59.952206 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.952172    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5kkg\" (UniqueName: \"kubernetes.io/projected/cabd37d3-8208-4220-80c4-2d28ed781ce3-kube-api-access-d5kkg\") pod \"payload-processing-7bb9489676-khhp6\" (UID: \"cabd37d3-8208-4220-80c4-2d28ed781ce3\") " pod="openshift-ingress/payload-processing-7bb9489676-khhp6"
Nov 04 03:27:59.952365 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.952263    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cabd37d3-8208-4220-80c4-2d28ed781ce3-plugins-config-volume\") pod \"payload-processing-7bb9489676-khhp6\" (UID: \"cabd37d3-8208-4220-80c4-2d28ed781ce3\") " pod="openshift-ingress/payload-processing-7bb9489676-khhp6"
Nov 04 03:27:59.953001 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:27:59.952980    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cabd37d3-8208-4220-80c4-2d28ed781ce3-plugins-config-volume\") pod \"payload-processing-7bb9489676-khhp6\" (UID: \"cabd37d3-8208-4220-80c4-2d28ed781ce3\") " pod="openshift-ingress/payload-processing-7bb9489676-khhp6"
Nov 04 03:28:00.013723 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:00.013691    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5kkg\" (UniqueName: \"kubernetes.io/projected/cabd37d3-8208-4220-80c4-2d28ed781ce3-kube-api-access-d5kkg\") pod \"payload-processing-7bb9489676-khhp6\" (UID: \"cabd37d3-8208-4220-80c4-2d28ed781ce3\") " pod="openshift-ingress/payload-processing-7bb9489676-khhp6"
Nov 04 03:28:00.068816 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:00.068779    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-khhp6"
Nov 04 03:28:00.279511 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:00.279487    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-chld2"]
Nov 04 03:28:00.282252 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:28:00.282217    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd770e48e_5587_4da6_838b_f7b52d18c86a.slice/crio-6696d15ddfde3fb4951d60bf242f219707b675a055a4f58f5c2ee0e3fd82f50a WatchSource:0}: Error finding container 6696d15ddfde3fb4951d60bf242f219707b675a055a4f58f5c2ee0e3fd82f50a: Status 404 returned error can't find the container with id 6696d15ddfde3fb4951d60bf242f219707b675a055a4f58f5c2ee0e3fd82f50a
Nov 04 03:28:00.439398 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:00.439370    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-khhp6"]
Nov 04 03:28:00.441703 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:28:00.441679    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcabd37d3_8208_4220_80c4_2d28ed781ce3.slice/crio-bf6bc4897fedf779fb3fe4639e2bfaf6456feab4f1e2b6b10bfd9e44fd4a4a4a WatchSource:0}: Error finding container bf6bc4897fedf779fb3fe4639e2bfaf6456feab4f1e2b6b10bfd9e44fd4a4a4a: Status 404 returned error can't find the container with id bf6bc4897fedf779fb3fe4639e2bfaf6456feab4f1e2b6b10bfd9e44fd4a4a4a
Nov 04 03:28:00.488179 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:00.488143    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-khhp6" event={"ID":"cabd37d3-8208-4220-80c4-2d28ed781ce3","Type":"ContainerStarted","Data":"bf6bc4897fedf779fb3fe4639e2bfaf6456feab4f1e2b6b10bfd9e44fd4a4a4a"}
Nov 04 03:28:00.489338 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:00.489311    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-chld2" event={"ID":"d770e48e-5587-4da6-838b-f7b52d18c86a","Type":"ContainerStarted","Data":"6696d15ddfde3fb4951d60bf242f219707b675a055a4f58f5c2ee0e3fd82f50a"}
Nov 04 03:28:03.503911 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:03.503876    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-khhp6" event={"ID":"cabd37d3-8208-4220-80c4-2d28ed781ce3","Type":"ContainerStarted","Data":"71c452cec7cb358393956dc3050d59d1e5cd83b54513f1824e275385204481f9"}
Nov 04 03:28:03.504289 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:03.504069    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bb9489676-khhp6"
Nov 04 03:28:03.505415 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:03.505390    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-chld2" event={"ID":"d770e48e-5587-4da6-838b-f7b52d18c86a","Type":"ContainerStarted","Data":"b7cb58def43f32d8feb02953719f67fb3c4d0d6e78e4e7c0eb3945c5e17b5a17"}
Nov 04 03:28:03.505519 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:03.505444    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-chld2"
Nov 04 03:28:03.621738 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:03.621633    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7bb9489676-khhp6" podStartSLOduration=1.650738115 podStartE2EDuration="4.621618149s" totalImagesPullingTime="2.970880034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:28:03.619818479 +0000 UTC m=+649.824418798" watchObservedRunningTime="2026-11-04 03:28:03.621618149 +0000 UTC m=+649.826218450"
Nov 04 03:28:03.737094 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:03.737045    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-chld2" podStartSLOduration=1.661638156 podStartE2EDuration="4.737032945s" totalImagesPullingTime="3.075394789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:28:03.736621513 +0000 UTC m=+649.941221816" watchObservedRunningTime="2026-11-04 03:28:03.737032945 +0000 UTC m=+649.941633247"
Nov 04 03:28:06.472555 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:06.472529    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"
Nov 04 03:28:06.827191 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:06.827155    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7cd9b8cccb-7bml5"]
Nov 04 03:28:06.830648 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:06.830627    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7cd9b8cccb-7bml5"
Nov 04 03:28:06.889076 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:06.889047    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7cd9b8cccb-7bml5"]
Nov 04 03:28:06.915906 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:06.915877    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8af2ef47-8b2a-4065-ad7f-09e70bed85e7-metrics-certs\") pod \"ai-gateway-controller-7cd9b8cccb-7bml5\" (UID: \"8af2ef47-8b2a-4065-ad7f-09e70bed85e7\") " pod="opendatahub/ai-gateway-controller-7cd9b8cccb-7bml5"
Nov 04 03:28:06.916097 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:06.915913    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6q4d\" (UniqueName: \"kubernetes.io/projected/8af2ef47-8b2a-4065-ad7f-09e70bed85e7-kube-api-access-m6q4d\") pod \"ai-gateway-controller-7cd9b8cccb-7bml5\" (UID: \"8af2ef47-8b2a-4065-ad7f-09e70bed85e7\") " pod="opendatahub/ai-gateway-controller-7cd9b8cccb-7bml5"
Nov 04 03:28:07.017304 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:07.017267    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8af2ef47-8b2a-4065-ad7f-09e70bed85e7-metrics-certs\") pod \"ai-gateway-controller-7cd9b8cccb-7bml5\" (UID: \"8af2ef47-8b2a-4065-ad7f-09e70bed85e7\") " pod="opendatahub/ai-gateway-controller-7cd9b8cccb-7bml5"
Nov 04 03:28:07.017304 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:07.017306    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6q4d\" (UniqueName: \"kubernetes.io/projected/8af2ef47-8b2a-4065-ad7f-09e70bed85e7-kube-api-access-m6q4d\") pod \"ai-gateway-controller-7cd9b8cccb-7bml5\" (UID: \"8af2ef47-8b2a-4065-ad7f-09e70bed85e7\") " pod="opendatahub/ai-gateway-controller-7cd9b8cccb-7bml5"
Nov 04 03:28:07.019681 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:07.019658    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8af2ef47-8b2a-4065-ad7f-09e70bed85e7-metrics-certs\") pod \"ai-gateway-controller-7cd9b8cccb-7bml5\" (UID: \"8af2ef47-8b2a-4065-ad7f-09e70bed85e7\") " pod="opendatahub/ai-gateway-controller-7cd9b8cccb-7bml5"
Nov 04 03:28:07.078865 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:07.078774    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6q4d\" (UniqueName: \"kubernetes.io/projected/8af2ef47-8b2a-4065-ad7f-09e70bed85e7-kube-api-access-m6q4d\") pod \"ai-gateway-controller-7cd9b8cccb-7bml5\" (UID: \"8af2ef47-8b2a-4065-ad7f-09e70bed85e7\") " pod="opendatahub/ai-gateway-controller-7cd9b8cccb-7bml5"
Nov 04 03:28:07.143573 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:07.143541    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7cd9b8cccb-7bml5"
Nov 04 03:28:07.514441 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:07.514293    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7cd9b8cccb-7bml5"]
Nov 04 03:28:07.517272 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:28:07.517241    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8af2ef47_8b2a_4065_ad7f_09e70bed85e7.slice/crio-c634a68b6319c9ee6a56d3a3927cc26b6c079aaf44843111e2d1f5da973d4026 WatchSource:0}: Error finding container c634a68b6319c9ee6a56d3a3927cc26b6c079aaf44843111e2d1f5da973d4026: Status 404 returned error can't find the container with id c634a68b6319c9ee6a56d3a3927cc26b6c079aaf44843111e2d1f5da973d4026
Nov 04 03:28:08.526628 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:08.526589    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7cd9b8cccb-7bml5" event={"ID":"8af2ef47-8b2a-4065-ad7f-09e70bed85e7","Type":"ContainerStarted","Data":"b755eb72a154569296fdae6421c6d90f7ac16167a25d29164de7d50eed3118e6"}
Nov 04 03:28:08.527034 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:08.526650    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7cd9b8cccb-7bml5" event={"ID":"8af2ef47-8b2a-4065-ad7f-09e70bed85e7","Type":"ContainerStarted","Data":"c634a68b6319c9ee6a56d3a3927cc26b6c079aaf44843111e2d1f5da973d4026"}
Nov 04 03:28:08.527034 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:08.526691    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7cd9b8cccb-7bml5"
Nov 04 03:28:08.644473 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:08.644419    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7cd9b8cccb-7bml5" podStartSLOduration=1.739117028 podStartE2EDuration="2.644403827s" totalImagesPullingTime="905.286799ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:28:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:28:08.643174117 +0000 UTC m=+654.847774436" watchObservedRunningTime="2026-11-04 03:28:08.644403827 +0000 UTC m=+654.849004129"
Nov 04 03:28:14.512399 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:14.512370    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-chld2"
Nov 04 03:28:14.512835 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:14.512443    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bb9489676-khhp6"
Nov 04 03:28:19.533970 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:19.533938    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7cd9b8cccb-7bml5"
Nov 04 03:28:19.703226 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:19.703191    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"]
Nov 04 03:28:19.703448 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:19.703403    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc" podUID="a12d5e79-b031-4add-8f08-27e6a4866876" containerName="manager" containerID="cri-o://cbdb2eb2c30ac78103924aa0a84aee8541b94cc708cab05b0fdfd92b73d764fb" gracePeriod=30
Nov 04 03:28:19.944677 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:19.944655    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"
Nov 04 03:28:20.025656 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:20.025629    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a12d5e79-b031-4add-8f08-27e6a4866876-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a12d5e79-b031-4add-8f08-27e6a4866876-metrics-certs\") pod \"a12d5e79-b031-4add-8f08-27e6a4866876\" (UID: \"a12d5e79-b031-4add-8f08-27e6a4866876\") "
Nov 04 03:28:20.025794 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:20.025697    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a12d5e79-b031-4add-8f08-27e6a4866876-kube-api-access-v9spc\" (UniqueName: \"kubernetes.io/projected/a12d5e79-b031-4add-8f08-27e6a4866876-kube-api-access-v9spc\") pod \"a12d5e79-b031-4add-8f08-27e6a4866876\" (UID: \"a12d5e79-b031-4add-8f08-27e6a4866876\") "
Nov 04 03:28:20.027898 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:20.027866    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a12d5e79-b031-4add-8f08-27e6a4866876-kube-api-access-v9spc" pod "a12d5e79-b031-4add-8f08-27e6a4866876" (UID: "a12d5e79-b031-4add-8f08-27e6a4866876"). InnerVolumeSpecName "kube-api-access-v9spc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:28:20.027999 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:20.027875    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a12d5e79-b031-4add-8f08-27e6a4866876-metrics-certs" pod "a12d5e79-b031-4add-8f08-27e6a4866876" (UID: "a12d5e79-b031-4add-8f08-27e6a4866876"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:28:20.126592 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:20.126519    4521 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a12d5e79-b031-4add-8f08-27e6a4866876-metrics-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:28:20.126592 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:20.126547    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9spc\" (UniqueName: \"kubernetes.io/projected/a12d5e79-b031-4add-8f08-27e6a4866876-kube-api-access-v9spc\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:28:20.575784 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:20.575748    4521 generic.go:356] "Generic (PLEG): container finished" podID="a12d5e79-b031-4add-8f08-27e6a4866876" containerID="cbdb2eb2c30ac78103924aa0a84aee8541b94cc708cab05b0fdfd92b73d764fb" exitCode=0
Nov 04 03:28:20.576298 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:20.575800    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"
Nov 04 03:28:20.576298 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:20.575809    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc" event={"ID":"a12d5e79-b031-4add-8f08-27e6a4866876","Type":"ContainerDied","Data":"cbdb2eb2c30ac78103924aa0a84aee8541b94cc708cab05b0fdfd92b73d764fb"}
Nov 04 03:28:20.576298 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:20.575864    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d8dddb676-jcvfc" event={"ID":"a12d5e79-b031-4add-8f08-27e6a4866876","Type":"ContainerDied","Data":"816f12b0d4bcc93313acd3928abc67b82a4e6b86bef2595238298634a2cee3e9"}
Nov 04 03:28:20.576298 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:20.575890    4521 scope.go:122] "RemoveContainer" containerID="cbdb2eb2c30ac78103924aa0a84aee8541b94cc708cab05b0fdfd92b73d764fb"
Nov 04 03:28:20.584362 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:20.584343    4521 scope.go:122] "RemoveContainer" containerID="cbdb2eb2c30ac78103924aa0a84aee8541b94cc708cab05b0fdfd92b73d764fb"
Nov 04 03:28:20.584627 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:28:20.584609    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbdb2eb2c30ac78103924aa0a84aee8541b94cc708cab05b0fdfd92b73d764fb\": container with ID starting with cbdb2eb2c30ac78103924aa0a84aee8541b94cc708cab05b0fdfd92b73d764fb not found: ID does not exist" containerID="cbdb2eb2c30ac78103924aa0a84aee8541b94cc708cab05b0fdfd92b73d764fb"
Nov 04 03:28:20.584688 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:20.584640    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbdb2eb2c30ac78103924aa0a84aee8541b94cc708cab05b0fdfd92b73d764fb"} err="failed to get container status \"cbdb2eb2c30ac78103924aa0a84aee8541b94cc708cab05b0fdfd92b73d764fb\": rpc error: code = NotFound desc = could not find container \"cbdb2eb2c30ac78103924aa0a84aee8541b94cc708cab05b0fdfd92b73d764fb\": container with ID starting with cbdb2eb2c30ac78103924aa0a84aee8541b94cc708cab05b0fdfd92b73d764fb not found: ID does not exist"
Nov 04 03:28:20.747289 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:20.747252    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"]
Nov 04 03:28:20.797443 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:20.797411    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-jcvfc"]
Nov 04 03:28:21.909545 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:21.909511    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a12d5e79-b031-4add-8f08-27e6a4866876" path="/var/lib/kubelet/pods/a12d5e79-b031-4add-8f08-27e6a4866876/volumes"
Nov 04 03:28:28.728580 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:28:28.728504    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:28:33.938279 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:33.938242    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"]
Nov 04 03:28:33.938749 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:33.938733    4521 cpu_manager.go:414] "containerMap: removing container" podUID="a12d5e79-b031-4add-8f08-27e6a4866876" containerName="manager"
Nov 04 03:28:33.938792 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:33.938752    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a12d5e79-b031-4add-8f08-27e6a4866876" containerName="manager"
Nov 04 03:28:33.938890 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:33.938878    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="a12d5e79-b031-4add-8f08-27e6a4866876" containerName="manager"
Nov 04 03:28:33.954768 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:33.954732    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.000763 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.000731    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"]
Nov 04 03:28:34.049687 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.049642    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 04 03:28:34.049687 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.049674    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-66tnw\""
Nov 04 03:28:34.049926 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.049642    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 04 03:28:34.056204 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.056183    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.056280 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.056228    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.056280 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.056256    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sqh7\" (UniqueName: \"kubernetes.io/projected/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-kube-api-access-8sqh7\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.056389 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.056377    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.056423 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.056399    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.056471 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.056461    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.058367 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.058348    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Nov 04 03:28:34.157176 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.157147    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.157359 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.157190    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sqh7\" (UniqueName: \"kubernetes.io/projected/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-kube-api-access-8sqh7\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.157359 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.157216    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.157359 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.157244    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.157359 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.157294    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.157359 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.157348    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.157648 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.157625    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.157700 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.157650    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.157700 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.157678    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.159618 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.159593    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.159738 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.159722    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.229914 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.229832    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sqh7\" (UniqueName: \"kubernetes.io/projected/f1d543fc-fedf-4c20-81b7-69f1b4689eb7-kube-api-access-8sqh7\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-dr574\" (UID: \"f1d543fc-fedf-4c20-81b7-69f1b4689eb7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.273003 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.272974    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:34.648195 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:34.648171    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"]
Nov 04 03:28:34.650696 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:28:34.650669    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1d543fc_fedf_4c20_81b7_69f1b4689eb7.slice/crio-0ca6ca66c874c03d66131d7e4a326ea42d43cf4852ab5400af66d50b29829cc3 WatchSource:0}: Error finding container 0ca6ca66c874c03d66131d7e4a326ea42d43cf4852ab5400af66d50b29829cc3: Status 404 returned error can't find the container with id 0ca6ca66c874c03d66131d7e4a326ea42d43cf4852ab5400af66d50b29829cc3
Nov 04 03:28:35.644731 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:35.644687    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574" event={"ID":"f1d543fc-fedf-4c20-81b7-69f1b4689eb7","Type":"ContainerStarted","Data":"0ca6ca66c874c03d66131d7e4a326ea42d43cf4852ab5400af66d50b29829cc3"}
Nov 04 03:28:37.655815 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:37.655774    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574" event={"ID":"f1d543fc-fedf-4c20-81b7-69f1b4689eb7","Type":"ContainerStarted","Data":"287a3dd2af6a4acfd971abb795dca257cb5a4bbccfe3fcfa08cadfb3f027419c"}
Nov 04 03:28:37.779072 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:37.779020    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574" podStartSLOduration=2.725359177 podStartE2EDuration="4.779005572s" totalImagesPullingTime="2.053646395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:28:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:28:37.777063545 +0000 UTC m=+683.981663867" watchObservedRunningTime="2026-11-04 03:28:37.779005572 +0000 UTC m=+683.983605874"
Nov 04 03:28:38.273355 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:38.273313    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:41.314297 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.314261    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"]
Nov 04 03:28:41.317965 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.317948    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.409803 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.409769    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0030652f-1a8f-42ff-a812-96f235cef9e7-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.409803 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.409805    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtv29\" (UniqueName: \"kubernetes.io/projected/0030652f-1a8f-42ff-a812-96f235cef9e7-kube-api-access-dtv29\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.410015 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.409932    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0030652f-1a8f-42ff-a812-96f235cef9e7-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.410015 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.409960    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0030652f-1a8f-42ff-a812-96f235cef9e7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.410015 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.409980    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0030652f-1a8f-42ff-a812-96f235cef9e7-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.410128 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.410084    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0030652f-1a8f-42ff-a812-96f235cef9e7-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.421357 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.421332    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Nov 04 03:28:41.422747 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.422726    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"]
Nov 04 03:28:41.511046 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.510984    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0030652f-1a8f-42ff-a812-96f235cef9e7-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.511046 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.511056    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0030652f-1a8f-42ff-a812-96f235cef9e7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.511297 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.511096    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0030652f-1a8f-42ff-a812-96f235cef9e7-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.511297 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.511131    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0030652f-1a8f-42ff-a812-96f235cef9e7-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.511297 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.511195    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0030652f-1a8f-42ff-a812-96f235cef9e7-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.511297 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.511219    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtv29\" (UniqueName: \"kubernetes.io/projected/0030652f-1a8f-42ff-a812-96f235cef9e7-kube-api-access-dtv29\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.511510 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.511487    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0030652f-1a8f-42ff-a812-96f235cef9e7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.511567 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.511526    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0030652f-1a8f-42ff-a812-96f235cef9e7-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.511663 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.511645    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0030652f-1a8f-42ff-a812-96f235cef9e7-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.513375 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.513355    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0030652f-1a8f-42ff-a812-96f235cef9e7-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.513607 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.513592    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0030652f-1a8f-42ff-a812-96f235cef9e7-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.572950 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.572874    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtv29\" (UniqueName: \"kubernetes.io/projected/0030652f-1a8f-42ff-a812-96f235cef9e7-kube-api-access-dtv29\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mgqz5\" (UID: \"0030652f-1a8f-42ff-a812-96f235cef9e7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:41.630903 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:41.630868    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:42.004955 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:42.004792    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"]
Nov 04 03:28:42.007161 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:28:42.007134    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0030652f_1a8f_42ff_a812_96f235cef9e7.slice/crio-8fddbfe9f7ceb033ce052e395d4eea241277068cd90d7b649993299242b11867 WatchSource:0}: Error finding container 8fddbfe9f7ceb033ce052e395d4eea241277068cd90d7b649993299242b11867: Status 404 returned error can't find the container with id 8fddbfe9f7ceb033ce052e395d4eea241277068cd90d7b649993299242b11867
Nov 04 03:28:42.684027 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:42.683991    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5" event={"ID":"0030652f-1a8f-42ff-a812-96f235cef9e7","Type":"ContainerStarted","Data":"8fddbfe9f7ceb033ce052e395d4eea241277068cd90d7b649993299242b11867"}
Nov 04 03:28:43.689733 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:43.689696    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5" event={"ID":"0030652f-1a8f-42ff-a812-96f235cef9e7","Type":"ContainerStarted","Data":"8fc419c92a452ab896e3b68bf14529b31ce179ebd459d7ff96ab3c90adbabdb9"}
Nov 04 03:28:43.809715 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:43.809655    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5" podStartSLOduration=2.197821319 podStartE2EDuration="2.809641513s" totalImagesPullingTime="611.820194ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:28:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:28:43.808118748 +0000 UTC m=+690.012719075" watchObservedRunningTime="2026-11-04 03:28:43.809641513 +0000 UTC m=+690.014241815"
Nov 04 03:28:44.274107 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:44.274069    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:44.286679 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:44.286647    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:44.631611 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:44.631537    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:44.704569 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:44.704535    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-dr574"
Nov 04 03:28:45.128689 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.128657    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"]
Nov 04 03:28:45.133282 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.133265    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.139408 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.139384    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh299\" (UniqueName: \"kubernetes.io/projected/dae99ce4-da89-4503-bc8f-dffb4f800846-kube-api-access-dh299\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.139529 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.139415    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dae99ce4-da89-4503-bc8f-dffb4f800846-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.139529 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.139454    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dae99ce4-da89-4503-bc8f-dffb4f800846-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.139649 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.139604    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dae99ce4-da89-4503-bc8f-dffb4f800846-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.139649 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.139640    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dae99ce4-da89-4503-bc8f-dffb4f800846-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.139765 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.139671    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dae99ce4-da89-4503-bc8f-dffb4f800846-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.186270 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.186238    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Nov 04 03:28:45.237973 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.237942    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"]
Nov 04 03:28:45.240297 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.240270    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh299\" (UniqueName: \"kubernetes.io/projected/dae99ce4-da89-4503-bc8f-dffb4f800846-kube-api-access-dh299\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.240297 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.240300    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dae99ce4-da89-4503-bc8f-dffb4f800846-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.240506 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.240331    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dae99ce4-da89-4503-bc8f-dffb4f800846-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.240506 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.240374    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dae99ce4-da89-4503-bc8f-dffb4f800846-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.240506 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.240388    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dae99ce4-da89-4503-bc8f-dffb4f800846-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.240506 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.240414    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dae99ce4-da89-4503-bc8f-dffb4f800846-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.240855 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.240821    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dae99ce4-da89-4503-bc8f-dffb4f800846-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.240984 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.240863    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dae99ce4-da89-4503-bc8f-dffb4f800846-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.240984 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.240932    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dae99ce4-da89-4503-bc8f-dffb4f800846-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.242821 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.242794    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dae99ce4-da89-4503-bc8f-dffb4f800846-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.243111 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.243086    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dae99ce4-da89-4503-bc8f-dffb4f800846-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.301937 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.301904    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh299\" (UniqueName: \"kubernetes.io/projected/dae99ce4-da89-4503-bc8f-dffb4f800846-kube-api-access-dh299\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r4s25\" (UID: \"dae99ce4-da89-4503-bc8f-dffb4f800846\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.446993 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.446914    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:45.823339 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:45.823268    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"]
Nov 04 03:28:45.825532 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:28:45.825504    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddae99ce4_da89_4503_bc8f_dffb4f800846.slice/crio-19788ed07fd52cefa561a1aaad6d8d64f99a07a61f5f3ed2c33701489a6aa434 WatchSource:0}: Error finding container 19788ed07fd52cefa561a1aaad6d8d64f99a07a61f5f3ed2c33701489a6aa434: Status 404 returned error can't find the container with id 19788ed07fd52cefa561a1aaad6d8d64f99a07a61f5f3ed2c33701489a6aa434
Nov 04 03:28:46.704345 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:46.704252    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25" event={"ID":"dae99ce4-da89-4503-bc8f-dffb4f800846","Type":"ContainerStarted","Data":"3578333a3e0fb6366d2da6f2b3663a04d13dc5da28f96675b236306324da8f4c"}
Nov 04 03:28:46.704345 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:46.704315    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25" event={"ID":"dae99ce4-da89-4503-bc8f-dffb4f800846","Type":"ContainerStarted","Data":"19788ed07fd52cefa561a1aaad6d8d64f99a07a61f5f3ed2c33701489a6aa434"}
Nov 04 03:28:46.822352 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:46.822299    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25" podStartSLOduration=1.2167995839999999 podStartE2EDuration="1.822285123s" totalImagesPullingTime="605.485539ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:28:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:28:46.82211704 +0000 UTC m=+693.026717374" watchObservedRunningTime="2026-11-04 03:28:46.822285123 +0000 UTC m=+693.026885424"
Nov 04 03:28:47.447777 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:47.447741    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:49.688759 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.688720    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"]
Nov 04 03:28:49.692355 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.692338    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.746117 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.746086    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Nov 04 03:28:49.752019 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.751987    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"]
Nov 04 03:28:49.779146 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.779105    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d0dd607-346c-478b-b28e-7738d56b325a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.779317 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.779158    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d0dd607-346c-478b-b28e-7738d56b325a-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.779317 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.779231    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d0dd607-346c-478b-b28e-7738d56b325a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.779434 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.779357    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d0dd607-346c-478b-b28e-7738d56b325a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.779434 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.779409    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d0dd607-346c-478b-b28e-7738d56b325a-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.779537 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.779438    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdcwf\" (UniqueName: \"kubernetes.io/projected/6d0dd607-346c-478b-b28e-7738d56b325a-kube-api-access-rdcwf\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.880958 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.880925    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d0dd607-346c-478b-b28e-7738d56b325a-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.881162 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.880965    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdcwf\" (UniqueName: \"kubernetes.io/projected/6d0dd607-346c-478b-b28e-7738d56b325a-kube-api-access-rdcwf\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.881162 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.881015    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d0dd607-346c-478b-b28e-7738d56b325a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.881162 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.881049    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d0dd607-346c-478b-b28e-7738d56b325a-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.881162 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.881086    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d0dd607-346c-478b-b28e-7738d56b325a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.881162 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.881128    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d0dd607-346c-478b-b28e-7738d56b325a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.881456 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.881433    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d0dd607-346c-478b-b28e-7738d56b325a-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.881516 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.881454    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d0dd607-346c-478b-b28e-7738d56b325a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.881550 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.881536    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d0dd607-346c-478b-b28e-7738d56b325a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.883357 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.883341    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d0dd607-346c-478b-b28e-7738d56b325a-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.883619 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.883598    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d0dd607-346c-478b-b28e-7738d56b325a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:49.954594 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:49.954513    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdcwf\" (UniqueName: \"kubernetes.io/projected/6d0dd607-346c-478b-b28e-7738d56b325a-kube-api-access-rdcwf\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-55lw7\" (UID: \"6d0dd607-346c-478b-b28e-7738d56b325a\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:50.005678 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:50.005640    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:50.379588 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:50.379486    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"]
Nov 04 03:28:50.382510 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:28:50.382480    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d0dd607_346c_478b_b28e_7738d56b325a.slice/crio-9cedf2383e65dfcffe6b552a7e6990eb39e03c03fcc793d6fedeb0e0851ea361 WatchSource:0}: Error finding container 9cedf2383e65dfcffe6b552a7e6990eb39e03c03fcc793d6fedeb0e0851ea361: Status 404 returned error can't find the container with id 9cedf2383e65dfcffe6b552a7e6990eb39e03c03fcc793d6fedeb0e0851ea361
Nov 04 03:28:50.723061 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:50.722975    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7" event={"ID":"6d0dd607-346c-478b-b28e-7738d56b325a","Type":"ContainerStarted","Data":"9cedf2383e65dfcffe6b552a7e6990eb39e03c03fcc793d6fedeb0e0851ea361"}
Nov 04 03:28:51.631879 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:51.631832    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:51.644313 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:51.644287    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:51.729374 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:51.729340    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7" event={"ID":"6d0dd607-346c-478b-b28e-7738d56b325a","Type":"ContainerStarted","Data":"d645eef204b14be5ae9f41801612aaf36907d6f187b3f999915655bd076011ef"}
Nov 04 03:28:51.740323 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:51.740302    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mgqz5"
Nov 04 03:28:51.903532 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:51.903418    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7" podStartSLOduration=2.127351724 podStartE2EDuration="2.903404547s" totalImagesPullingTime="776.052823ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:28:51.90146588 +0000 UTC m=+698.106066173" watchObservedRunningTime="2026-11-04 03:28:51.903404547 +0000 UTC m=+698.108004849"
Nov 04 03:28:52.005808 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:52.005779    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:28:53.204913 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.204870    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"]
Nov 04 03:28:53.208878 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.208839    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.312120 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.312087    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkb6b\" (UniqueName: \"kubernetes.io/projected/61e73af7-b506-42f5-8594-8ad1387e39d5-kube-api-access-hkb6b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.312280 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.312135    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61e73af7-b506-42f5-8594-8ad1387e39d5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.312320 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.312283    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61e73af7-b506-42f5-8594-8ad1387e39d5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.312359 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.312345    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61e73af7-b506-42f5-8594-8ad1387e39d5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.312389 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.312359    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Nov 04 03:28:53.312421 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.312406    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61e73af7-b506-42f5-8594-8ad1387e39d5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.312452 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.312441    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61e73af7-b506-42f5-8594-8ad1387e39d5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.314016 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.313997    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"]
Nov 04 03:28:53.413254 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.413221    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61e73af7-b506-42f5-8594-8ad1387e39d5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.413438 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.413273    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61e73af7-b506-42f5-8594-8ad1387e39d5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.413438 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.413312    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61e73af7-b506-42f5-8594-8ad1387e39d5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.413438 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.413348    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61e73af7-b506-42f5-8594-8ad1387e39d5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.413574 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.413485    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkb6b\" (UniqueName: \"kubernetes.io/projected/61e73af7-b506-42f5-8594-8ad1387e39d5-kube-api-access-hkb6b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.413574 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.413543    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61e73af7-b506-42f5-8594-8ad1387e39d5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.413710 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.413687    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61e73af7-b506-42f5-8594-8ad1387e39d5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.413772 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.413711    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61e73af7-b506-42f5-8594-8ad1387e39d5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.413981 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.413957    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61e73af7-b506-42f5-8594-8ad1387e39d5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.415626 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.415603    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61e73af7-b506-42f5-8594-8ad1387e39d5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.415952 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.415931    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61e73af7-b506-42f5-8594-8ad1387e39d5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.476836 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.476767    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkb6b\" (UniqueName: \"kubernetes.io/projected/61e73af7-b506-42f5-8594-8ad1387e39d5-kube-api-access-hkb6b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49\" (UID: \"61e73af7-b506-42f5-8594-8ad1387e39d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.524903 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.524874    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:53.898523 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:53.898496    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"]
Nov 04 03:28:53.901168 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:28:53.901141    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61e73af7_b506_42f5_8594_8ad1387e39d5.slice/crio-a97dda457855e800e71a4f741a70944e3fcc8e518a4af39d2f7a685011f98a29 WatchSource:0}: Error finding container a97dda457855e800e71a4f741a70944e3fcc8e518a4af39d2f7a685011f98a29: Status 404 returned error can't find the container with id a97dda457855e800e71a4f741a70944e3fcc8e518a4af39d2f7a685011f98a29
Nov 04 03:28:54.744733 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:54.744698    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49" event={"ID":"61e73af7-b506-42f5-8594-8ad1387e39d5","Type":"ContainerStarted","Data":"814fbf8e69ce8f98448636f20da24828f6c94f6a3d0afcf69368dbc42eab8289"}
Nov 04 03:28:54.745129 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:54.744750    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49" event={"ID":"61e73af7-b506-42f5-8594-8ad1387e39d5","Type":"ContainerStarted","Data":"a97dda457855e800e71a4f741a70944e3fcc8e518a4af39d2f7a685011f98a29"}
Nov 04 03:28:54.864334 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:54.864195    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49" podStartSLOduration=1.221911086 podStartE2EDuration="1.864181239s" totalImagesPullingTime="642.270153ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:28:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:28:54.863111047 +0000 UTC m=+701.067711368" watchObservedRunningTime="2026-11-04 03:28:54.864181239 +0000 UTC m=+701.068781541"
Nov 04 03:28:55.447346 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:55.447310    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:55.459874 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:55.459829    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:55.525093 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:55.525063    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:28:55.760433 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:28:55.760357    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r4s25"
Nov 04 03:28:58.728777 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:28:58.728744    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:29:00.006170 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:00.006138    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:29:00.018741 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:00.018712    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:29:00.792965 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:00.792931    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-55lw7"
Nov 04 03:29:02.097140 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.097107    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"]
Nov 04 03:29:02.105965 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.105933    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.200080 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.200048    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bee1a355-b980-49a4-acbc-3527d6107ddc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.200080 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.200082    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bee1a355-b980-49a4-acbc-3527d6107ddc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.200325 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.200097    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bee1a355-b980-49a4-acbc-3527d6107ddc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.200325 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.200128    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bee1a355-b980-49a4-acbc-3527d6107ddc-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.200325 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.200295    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bee1a355-b980-49a4-acbc-3527d6107ddc-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.200468 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.200340    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff6xh\" (UniqueName: \"kubernetes.io/projected/bee1a355-b980-49a4-acbc-3527d6107ddc-kube-api-access-ff6xh\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.209153 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.209129    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Nov 04 03:29:02.210670 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.210650    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"]
Nov 04 03:29:02.301107 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.301069    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff6xh\" (UniqueName: \"kubernetes.io/projected/bee1a355-b980-49a4-acbc-3527d6107ddc-kube-api-access-ff6xh\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.301295 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.301161    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bee1a355-b980-49a4-acbc-3527d6107ddc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.301295 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.301180    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bee1a355-b980-49a4-acbc-3527d6107ddc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.301295 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.301194    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bee1a355-b980-49a4-acbc-3527d6107ddc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.301295 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.301210    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bee1a355-b980-49a4-acbc-3527d6107ddc-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.301295 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.301235    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bee1a355-b980-49a4-acbc-3527d6107ddc-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.301615 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.301593    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bee1a355-b980-49a4-acbc-3527d6107ddc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.301683 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.301664    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bee1a355-b980-49a4-acbc-3527d6107ddc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.301725 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.301689    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bee1a355-b980-49a4-acbc-3527d6107ddc-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.303625 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.303605    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bee1a355-b980-49a4-acbc-3527d6107ddc-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.303751 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.303733    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bee1a355-b980-49a4-acbc-3527d6107ddc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.362657 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.362560    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"]
Nov 04 03:29:02.364991 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.364970    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff6xh\" (UniqueName: \"kubernetes.io/projected/bee1a355-b980-49a4-acbc-3527d6107ddc-kube-api-access-ff6xh\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t\" (UID: \"bee1a355-b980-49a4-acbc-3527d6107ddc\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.366455 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.366438    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.401812 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.401776    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.401955 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.401822    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jltks\" (UniqueName: \"kubernetes.io/projected/0c64e234-343b-42ac-a4f1-44267416b6c3-kube-api-access-jltks\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.401955 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.401943    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.402047 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.401989    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.402084 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.402057    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.402130 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.402092    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.402210 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.402194    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.402270 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.402240    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.402312 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.402299    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c64e234-343b-42ac-a4f1-44267416b6c3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.419202 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.419172    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:02.419346 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.419324    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 04 03:29:02.419484 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.419467    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-dk4hf\""
Nov 04 03:29:02.429172 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.429050    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"]
Nov 04 03:29:02.503415 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.503385    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c64e234-343b-42ac-a4f1-44267416b6c3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.503790 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.503425    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.503790 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.503462    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jltks\" (UniqueName: \"kubernetes.io/projected/0c64e234-343b-42ac-a4f1-44267416b6c3-kube-api-access-jltks\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.503790 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.503514    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.503790 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.503562    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.503790 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.503604    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.503790 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.503648    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.503790 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.503702    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.503790 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.503739    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.504304 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.503821    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.504304 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.504045    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.504304 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.504156    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.504304 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.504274    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c64e234-343b-42ac-a4f1-44267416b6c3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.504497 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.504482    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.506401 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.506359    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.507134 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.507112    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.564816 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.564789    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.564950 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.564884    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jltks\" (UniqueName: \"kubernetes.io/projected/0c64e234-343b-42ac-a4f1-44267416b6c3-kube-api-access-jltks\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-bvxs5\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.685654 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.685592    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:02.798875 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:02.798818    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"]
Nov 04 03:29:02.801743 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:29:02.801720    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbee1a355_b980_49a4_acbc_3527d6107ddc.slice/crio-c98cd3c8036897bb528f2092267a04d86eafcad86dd5b75af2d120ce99ea9076 WatchSource:0}: Error finding container c98cd3c8036897bb528f2092267a04d86eafcad86dd5b75af2d120ce99ea9076: Status 404 returned error can't find the container with id c98cd3c8036897bb528f2092267a04d86eafcad86dd5b75af2d120ce99ea9076
Nov 04 03:29:03.064563 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:03.064537    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"]
Nov 04 03:29:03.066986 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:29:03.066957    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c64e234_343b_42ac_a4f1_44267416b6c3.slice/crio-1688c937bf89398aa8b5b11f08ad8480fd8e96c60729f6c8e6881c078422e72c WatchSource:0}: Error finding container 1688c937bf89398aa8b5b11f08ad8480fd8e96c60729f6c8e6881c078422e72c: Status 404 returned error can't find the container with id 1688c937bf89398aa8b5b11f08ad8480fd8e96c60729f6c8e6881c078422e72c
Nov 04 03:29:03.525580 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:03.525548    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:29:03.537827 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:03.537801    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:29:03.796678 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:03.796646    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t" event={"ID":"bee1a355-b980-49a4-acbc-3527d6107ddc","Type":"ContainerStarted","Data":"311e8e3d77b70fed613455b518a9c0a2218685e8610359eb9f09292c3e2b87d9"}
Nov 04 03:29:03.796883 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:03.796705    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t" event={"ID":"bee1a355-b980-49a4-acbc-3527d6107ddc","Type":"ContainerStarted","Data":"c98cd3c8036897bb528f2092267a04d86eafcad86dd5b75af2d120ce99ea9076"}
Nov 04 03:29:03.797892 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:03.797871    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5" event={"ID":"0c64e234-343b-42ac-a4f1-44267416b6c3","Type":"ContainerStarted","Data":"1688c937bf89398aa8b5b11f08ad8480fd8e96c60729f6c8e6881c078422e72c"}
Nov 04 03:29:03.808956 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:03.808937    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49"
Nov 04 03:29:03.915659 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:03.915598    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t" podStartSLOduration=1.310891675 podStartE2EDuration="1.915580614s" totalImagesPullingTime="604.688939ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:29:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:29:03.9145742 +0000 UTC m=+710.119174503" watchObservedRunningTime="2026-11-04 03:29:03.915580614 +0000 UTC m=+710.120180919"
Nov 04 03:29:04.420156 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:04.420120    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:07.215051 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:07.215016    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:29:07.215318 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:07.215084    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:29:07.215318 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:07.215113    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:29:07.822420 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:07.822372    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5" event={"ID":"0c64e234-343b-42ac-a4f1-44267416b6c3","Type":"ContainerStarted","Data":"9245d123ade9a6a83bd4ca9925fee32232af700552de4001414ef4ab879c0624"}
Nov 04 03:29:08.007808 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.007739    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5" podStartSLOduration=1.8626772520000001 podStartE2EDuration="6.007718726s" totalImagesPullingTime="4.145041474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:29:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:29:08.007159817 +0000 UTC m=+714.211760120" watchObservedRunningTime="2026-11-04 03:29:08.007718726 +0000 UTC m=+714.212319029"
Nov 04 03:29:08.326027 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.325985    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"]
Nov 04 03:29:08.329742 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.329726    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.382791 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.382764    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Nov 04 03:29:08.415224 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.415181    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"]
Nov 04 03:29:08.463235 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.463191    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.463421 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.463313    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.463421 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.463372    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67j2l\" (UniqueName: \"kubernetes.io/projected/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-kube-api-access-67j2l\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.463421 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.463398    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.463529 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.463470    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.463576 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.463563    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.565037 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.565002    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.565037 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.565047    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.565304 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.565098    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.565304 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.565118    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67j2l\" (UniqueName: \"kubernetes.io/projected/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-kube-api-access-67j2l\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.565304 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.565139    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.565304 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.565161    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.565505 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.565459    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.565560 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.565525    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.565643 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.565617    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.567428 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.567407    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.567668 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.567651    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.627520 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.627438    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67j2l\" (UniqueName: \"kubernetes.io/projected/9c4b4d5b-04ce-4860-90e2-3161029b2ca0-kube-api-access-67j2l\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv\" (UID: \"9c4b4d5b-04ce-4860-90e2-3161029b2ca0\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.641187 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.641158    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:08.698235 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.697717    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:08.698235 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.697906    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 04 03:29:08.698235 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:08.697950    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5" podUID="0c64e234-343b-42ac-a4f1-44267416b6c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 04 03:29:09.038146 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:09.038119    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"]
Nov 04 03:29:09.039540 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:29:09.039513    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c4b4d5b_04ce_4860_90e2_3161029b2ca0.slice/crio-2327507ac95d761e568513e5c2c6d11c61cc639cc1b36c75055d5a39239815b2 WatchSource:0}: Error finding container 2327507ac95d761e568513e5c2c6d11c61cc639cc1b36c75055d5a39239815b2: Status 404 returned error can't find the container with id 2327507ac95d761e568513e5c2c6d11c61cc639cc1b36c75055d5a39239815b2
Nov 04 03:29:09.709878 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:09.706089    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 04 03:29:09.709878 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:09.706150    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5" podUID="0c64e234-343b-42ac-a4f1-44267416b6c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 04 03:29:09.833732 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:09.833691    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv" event={"ID":"9c4b4d5b-04ce-4860-90e2-3161029b2ca0","Type":"ContainerStarted","Data":"b6fa492c60d5149616568a6222b3837aaaaa5f359b6de2ab88db1d5617eac238"}
Nov 04 03:29:09.833942 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:09.833747    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv" event={"ID":"9c4b4d5b-04ce-4860-90e2-3161029b2ca0","Type":"ContainerStarted","Data":"2327507ac95d761e568513e5c2c6d11c61cc639cc1b36c75055d5a39239815b2"}
Nov 04 03:29:09.978670 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:09.978570    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv" podStartSLOduration=1.339411787 podStartE2EDuration="1.978554989s" totalImagesPullingTime="639.143202ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:29:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:29:09.976677552 +0000 UTC m=+716.181277873" watchObservedRunningTime="2026-11-04 03:29:09.978554989 +0000 UTC m=+716.183155291"
Nov 04 03:29:10.641288 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:10.641248    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:10.704869 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:10.703285    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 04 03:29:10.704869 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:10.703350    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5" podUID="0c64e234-343b-42ac-a4f1-44267416b6c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 04 03:29:11.705869 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:11.705262    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 04 03:29:11.705869 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:11.705324    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5" podUID="0c64e234-343b-42ac-a4f1-44267416b6c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 04 03:29:12.420312 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:12.420273    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:12.432497 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:12.432471    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:12.709927 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:12.706004    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 04 03:29:12.709927 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:12.706067    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5" podUID="0c64e234-343b-42ac-a4f1-44267416b6c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 04 03:29:12.858376 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:12.858343    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t"
Nov 04 03:29:13.686201 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:13.686170    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 04 03:29:13.686376 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:13.686228    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5" podUID="0c64e234-343b-42ac-a4f1-44267416b6c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 04 03:29:15.492560 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:15.492513    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:15.493040 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:15.492918    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:15.493822 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:15.493803    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:29:18.641807 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:18.641769    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:18.654084 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:18.654062    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:18.894866 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:18.894782    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv"
Nov 04 03:29:28.729943 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:29:28.729901    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:29:32.615263 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.615228    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"]
Nov 04 03:29:32.618617 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.618597    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.710557 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.710520    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/beb6c720-f8d2-4337-8faa-c9e5e6414d33-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.710739 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.710570    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sb4v\" (UniqueName: \"kubernetes.io/projected/beb6c720-f8d2-4337-8faa-c9e5e6414d33-kube-api-access-9sb4v\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.710739 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.710591    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/beb6c720-f8d2-4337-8faa-c9e5e6414d33-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.710739 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.710702    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/beb6c720-f8d2-4337-8faa-c9e5e6414d33-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.710879 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.710742    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/beb6c720-f8d2-4337-8faa-c9e5e6414d33-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.710879 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.710770    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/beb6c720-f8d2-4337-8faa-c9e5e6414d33-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.710879 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.710869    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/beb6c720-f8d2-4337-8faa-c9e5e6414d33-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.710976 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.710916    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/beb6c720-f8d2-4337-8faa-c9e5e6414d33-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.710976 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.710966    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/beb6c720-f8d2-4337-8faa-c9e5e6414d33-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.756722 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.756691    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"]
Nov 04 03:29:32.812091 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.812063    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/beb6c720-f8d2-4337-8faa-c9e5e6414d33-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.812283 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.812096    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/beb6c720-f8d2-4337-8faa-c9e5e6414d33-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.812283 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.812132    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/beb6c720-f8d2-4337-8faa-c9e5e6414d33-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.812283 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.812180    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/beb6c720-f8d2-4337-8faa-c9e5e6414d33-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.812283 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.812216    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/beb6c720-f8d2-4337-8faa-c9e5e6414d33-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.812283 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.812277    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/beb6c720-f8d2-4337-8faa-c9e5e6414d33-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.812560 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.812327    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sb4v\" (UniqueName: \"kubernetes.io/projected/beb6c720-f8d2-4337-8faa-c9e5e6414d33-kube-api-access-9sb4v\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.812560 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.812359    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/beb6c720-f8d2-4337-8faa-c9e5e6414d33-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.812560 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.812396    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/beb6c720-f8d2-4337-8faa-c9e5e6414d33-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.812560 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.812529    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/beb6c720-f8d2-4337-8faa-c9e5e6414d33-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.812771 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.812619    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/beb6c720-f8d2-4337-8faa-c9e5e6414d33-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.812891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.812870    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/beb6c720-f8d2-4337-8faa-c9e5e6414d33-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.812974 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.812954    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/beb6c720-f8d2-4337-8faa-c9e5e6414d33-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.813070 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.813054    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/beb6c720-f8d2-4337-8faa-c9e5e6414d33-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.814406 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.814376    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/beb6c720-f8d2-4337-8faa-c9e5e6414d33-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.814647 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.814630    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/beb6c720-f8d2-4337-8faa-c9e5e6414d33-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.874711 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.874642    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/beb6c720-f8d2-4337-8faa-c9e5e6414d33-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.874872 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.874720    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sb4v\" (UniqueName: \"kubernetes.io/projected/beb6c720-f8d2-4337-8faa-c9e5e6414d33-kube-api-access-9sb4v\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-st9jq\" (UID: \"beb6c720-f8d2-4337-8faa-c9e5e6414d33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:32.931853 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:32.931809    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:33.365445 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:33.365417    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"]
Nov 04 03:29:33.367402 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:29:33.367371    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeb6c720_f8d2_4337_8faa_c9e5e6414d33.slice/crio-16c2ddead2969495a509b0e0723b4d079713d9680b8bd88a453ffcfffcf970df WatchSource:0}: Error finding container 16c2ddead2969495a509b0e0723b4d079713d9680b8bd88a453ffcfffcf970df: Status 404 returned error can't find the container with id 16c2ddead2969495a509b0e0723b4d079713d9680b8bd88a453ffcfffcf970df
Nov 04 03:29:33.369601 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:33.369565    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:29:33.369717 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:33.369666    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:29:33.369777 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:33.369714    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:29:33.954081 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:33.954045    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq" event={"ID":"beb6c720-f8d2-4337-8faa-c9e5e6414d33","Type":"ContainerStarted","Data":"c3eb366980b37c330e78abf4b81074006784b19ce33bed189b8f08ebf8a57706"}
Nov 04 03:29:33.954602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:33.954095    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq" event={"ID":"beb6c720-f8d2-4337-8faa-c9e5e6414d33","Type":"ContainerStarted","Data":"16c2ddead2969495a509b0e0723b4d079713d9680b8bd88a453ffcfffcf970df"}
Nov 04 03:29:34.129330 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:34.129281    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq" podStartSLOduration=2.129267578 podStartE2EDuration="2.129267578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:29:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:29:34.123601629 +0000 UTC m=+740.328201976" watchObservedRunningTime="2026-11-04 03:29:34.129267578 +0000 UTC m=+740.333867879"
Nov 04 03:29:34.932391 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:34.932360    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:34.933918 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:34.933896    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-st9jq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 04 03:29:34.933991 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:34.933976    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq" podUID="beb6c720-f8d2-4337-8faa-c9e5e6414d33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 04 03:29:35.932806 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:35.932773    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-st9jq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 04 03:29:35.933208 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:35.932831    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq" podUID="beb6c720-f8d2-4337-8faa-c9e5e6414d33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 04 03:29:36.932749 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:36.932709    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-st9jq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 04 03:29:36.932980 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:36.932768    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq" podUID="beb6c720-f8d2-4337-8faa-c9e5e6414d33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 04 03:29:37.932623 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:37.932592    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-st9jq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 04 03:29:37.932793 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:37.932644    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq" podUID="beb6c720-f8d2-4337-8faa-c9e5e6414d33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 04 03:29:38.932341 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:38.932297    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-st9jq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 04 03:29:38.932753 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:38.932356    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq" podUID="beb6c720-f8d2-4337-8faa-c9e5e6414d33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 04 03:29:39.932735 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:39.932693    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-st9jq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 04 03:29:39.933132 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:39.932759    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq" podUID="beb6c720-f8d2-4337-8faa-c9e5e6414d33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 04 03:29:40.932472 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:40.932435    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-st9jq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 04 03:29:40.932645 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:40.932491    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq" podUID="beb6c720-f8d2-4337-8faa-c9e5e6414d33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 04 03:29:41.936758 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:41.936729    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:41.937267 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:41.937127    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:41.937806 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:29:41.937787    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-st9jq"
Nov 04 03:29:58.729356 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:29:58.729279    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:30:28.729451 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:30:28.729410    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:30:56.008169 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.008134    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"]
Nov 04 03:30:56.011689 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.011672    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"
Nov 04 03:30:56.067663 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.067632    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-fba151\""
Nov 04 03:30:56.110485 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.110451    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rpln\" (UniqueName: \"kubernetes.io/projected/52ad1670-708f-4b8d-8d2d-4c5101f71319-kube-api-access-2rpln\") pod \"payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b\" (UID: \"52ad1670-708f-4b8d-8d2d-4c5101f71319\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"
Nov 04 03:30:56.110673 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.110601    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52ad1670-708f-4b8d-8d2d-4c5101f71319-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b\" (UID: \"52ad1670-708f-4b8d-8d2d-4c5101f71319\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"
Nov 04 03:30:56.118321 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.118294    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-fba151-dockercfg-mwt6x\""
Nov 04 03:30:56.120056 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.120033    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"]
Nov 04 03:30:56.171855 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.171801    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"]
Nov 04 03:30:56.176085 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.176061    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"
Nov 04 03:30:56.211739 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.211706    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/508106f1-2ca8-4a87-aa92-49d54846e38b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7\" (UID: \"508106f1-2ca8-4a87-aa92-49d54846e38b\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"
Nov 04 03:30:56.211936 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.211781    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rpln\" (UniqueName: \"kubernetes.io/projected/52ad1670-708f-4b8d-8d2d-4c5101f71319-kube-api-access-2rpln\") pod \"payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b\" (UID: \"52ad1670-708f-4b8d-8d2d-4c5101f71319\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"
Nov 04 03:30:56.211936 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.211900    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctwhx\" (UniqueName: \"kubernetes.io/projected/508106f1-2ca8-4a87-aa92-49d54846e38b-kube-api-access-ctwhx\") pod \"payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7\" (UID: \"508106f1-2ca8-4a87-aa92-49d54846e38b\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"
Nov 04 03:30:56.212071 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.211999    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52ad1670-708f-4b8d-8d2d-4c5101f71319-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b\" (UID: \"52ad1670-708f-4b8d-8d2d-4c5101f71319\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"
Nov 04 03:30:56.212901 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.212870    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52ad1670-708f-4b8d-8d2d-4c5101f71319-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b\" (UID: \"52ad1670-708f-4b8d-8d2d-4c5101f71319\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"
Nov 04 03:30:56.219389 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.219346    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"]
Nov 04 03:30:56.273542 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.273512    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rpln\" (UniqueName: \"kubernetes.io/projected/52ad1670-708f-4b8d-8d2d-4c5101f71319-kube-api-access-2rpln\") pod \"payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b\" (UID: \"52ad1670-708f-4b8d-8d2d-4c5101f71319\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"
Nov 04 03:30:56.312533 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.312492    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctwhx\" (UniqueName: \"kubernetes.io/projected/508106f1-2ca8-4a87-aa92-49d54846e38b-kube-api-access-ctwhx\") pod \"payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7\" (UID: \"508106f1-2ca8-4a87-aa92-49d54846e38b\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"
Nov 04 03:30:56.312698 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.312626    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/508106f1-2ca8-4a87-aa92-49d54846e38b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7\" (UID: \"508106f1-2ca8-4a87-aa92-49d54846e38b\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"
Nov 04 03:30:56.313328 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.313308    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/508106f1-2ca8-4a87-aa92-49d54846e38b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7\" (UID: \"508106f1-2ca8-4a87-aa92-49d54846e38b\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"
Nov 04 03:30:56.324421 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.324391    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"
Nov 04 03:30:56.393926 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.393827    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctwhx\" (UniqueName: \"kubernetes.io/projected/508106f1-2ca8-4a87-aa92-49d54846e38b-kube-api-access-ctwhx\") pod \"payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7\" (UID: \"508106f1-2ca8-4a87-aa92-49d54846e38b\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"
Nov 04 03:30:56.489041 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.489004    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"
Nov 04 03:30:56.698457 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.698433    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"]
Nov 04 03:30:56.701063 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:30:56.701036    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52ad1670_708f_4b8d_8d2d_4c5101f71319.slice/crio-ed4dd8251fdf2a1290b91924297c42631b17f7fffa0eb28e68562a8e0b7d5350 WatchSource:0}: Error finding container ed4dd8251fdf2a1290b91924297c42631b17f7fffa0eb28e68562a8e0b7d5350: Status 404 returned error can't find the container with id ed4dd8251fdf2a1290b91924297c42631b17f7fffa0eb28e68562a8e0b7d5350
Nov 04 03:30:56.861189 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:56.861165    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"]
Nov 04 03:30:56.863631 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:30:56.863603    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod508106f1_2ca8_4a87_aa92_49d54846e38b.slice/crio-26ad7074966720ae8051affd15b2399460820efa4d9d66db6485653d1ce7b2d9 WatchSource:0}: Error finding container 26ad7074966720ae8051affd15b2399460820efa4d9d66db6485653d1ce7b2d9: Status 404 returned error can't find the container with id 26ad7074966720ae8051affd15b2399460820efa4d9d66db6485653d1ce7b2d9
Nov 04 03:30:57.338299 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.338257    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7" event={"ID":"508106f1-2ca8-4a87-aa92-49d54846e38b","Type":"ContainerStarted","Data":"ed779ff59049849edd9c060251a4be29c6dc3d58888fd9b49786a4dc6d6a7fc5"}
Nov 04 03:30:57.338754 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.338322    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"
Nov 04 03:30:57.338754 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.338344    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7" event={"ID":"508106f1-2ca8-4a87-aa92-49d54846e38b","Type":"ContainerStarted","Data":"26ad7074966720ae8051affd15b2399460820efa4d9d66db6485653d1ce7b2d9"}
Nov 04 03:30:57.339830 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.339805    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b" event={"ID":"52ad1670-708f-4b8d-8d2d-4c5101f71319","Type":"ContainerStarted","Data":"4def60bed6cb81c9823da286d4538e61651adf10322c968b3b9d2f6ecf452a3b"}
Nov 04 03:30:57.339956 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.339861    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b" event={"ID":"52ad1670-708f-4b8d-8d2d-4c5101f71319","Type":"ContainerStarted","Data":"ed4dd8251fdf2a1290b91924297c42631b17f7fffa0eb28e68562a8e0b7d5350"}
Nov 04 03:30:57.339956 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.339890    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"
Nov 04 03:30:57.457058 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.456987    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7" podStartSLOduration=1.456969327 podStartE2EDuration="1.456969327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:30:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:30:57.456634192 +0000 UTC m=+823.661234493" watchObservedRunningTime="2026-11-04 03:30:57.456969327 +0000 UTC m=+823.661569630"
Nov 04 03:30:57.573759 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.573704    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b" podStartSLOduration=2.573688244 podStartE2EDuration="2.573688244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:30:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:30:57.572692166 +0000 UTC m=+823.777292500" watchObservedRunningTime="2026-11-04 03:30:57.573688244 +0000 UTC m=+823.778288547"
Nov 04 03:30:57.831357 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.831324    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"]
Nov 04 03:30:57.835136 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.835116    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:57.888091 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.888058    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-50429e-openshift-default-dockercfg-96qjd\""
Nov 04 03:30:57.897190 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.897156    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"]
Nov 04 03:30:57.928207 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.928173    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9qd7\" (UniqueName: \"kubernetes.io/projected/78ed61dd-e0d5-4165-ad93-58d761dfdd00-kube-api-access-d9qd7\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:57.928374 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.928224    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-envoy\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:57.928374 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.928298    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-data\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:57.928374 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.928360    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-workload-socket\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:57.928535 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.928493    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-credential-socket\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:57.928587 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.928546    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istiod-ca-cert\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:57.928627 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.928605    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-workload-certs\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:57.928678 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.928661    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-token\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:57.928725 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:57.928711    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-podinfo\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.029761 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.029714    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-credential-socket\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.029989 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.029784    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istiod-ca-cert\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.029989 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.029823    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-workload-certs\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.029989 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.029896    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-token\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.029989 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.029940    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-podinfo\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.030239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.029987    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9qd7\" (UniqueName: \"kubernetes.io/projected/78ed61dd-e0d5-4165-ad93-58d761dfdd00-kube-api-access-d9qd7\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.030239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.030048    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-envoy\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.030239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.030080    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-data\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.030239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.030145    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-workload-socket\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.030458 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.030236    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-credential-socket\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.030633 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.030607    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-workload-certs\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.030736 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.030706    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-data\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.030893 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.030751    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-workload-socket\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.030893 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.030883    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istiod-ca-cert\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.032414 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.032389    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-envoy\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.032685 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.032667    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-podinfo\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.091524 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.091459    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-token\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.091655 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.091599    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9qd7\" (UniqueName: \"kubernetes.io/projected/78ed61dd-e0d5-4165-ad93-58d761dfdd00-kube-api-access-d9qd7\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.149252 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.149219    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:30:58.561203 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.561160    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"]
Nov 04 03:30:58.565739 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.565722    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:30:58.592407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.592379    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"]
Nov 04 03:30:58.614172 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:30:58.614148    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78ed61dd_e0d5_4165_ad93_58d761dfdd00.slice/crio-61a1e77260be0ae7f1103f6799e9f8ddff6d55781d3ebabab2df28c15fe37aca WatchSource:0}: Error finding container 61a1e77260be0ae7f1103f6799e9f8ddff6d55781d3ebabab2df28c15fe37aca: Status 404 returned error can't find the container with id 61a1e77260be0ae7f1103f6799e9f8ddff6d55781d3ebabab2df28c15fe37aca
Nov 04 03:30:58.616461 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.616433    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:30:58.616521 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.616490    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:30:58.616556 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.616520    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:30:58.618599 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.618527    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-82c724-serving-cert\""
Nov 04 03:30:58.624751 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.624728    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"]
Nov 04 03:30:58.637452 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.637428    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4d61e92-8c91-4bea-9811-4535ae9f00ef-maas-api-tls\") pod \"maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg\" (UID: \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:30:58.637574 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.637558    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4d61e92-8c91-4bea-9811-4535ae9f00ef-metrics-certs\") pod \"maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg\" (UID: \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:30:58.637645 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.637616    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d61e92-8c91-4bea-9811-4535ae9f00ef-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg\" (UID: \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:30:58.637645 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.637635    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdnzt\" (UniqueName: \"kubernetes.io/projected/c4d61e92-8c91-4bea-9811-4535ae9f00ef-kube-api-access-hdnzt\") pod \"maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg\" (UID: \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:30:58.737456 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:30:58.737419    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:30:58.738859 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.738819    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4d61e92-8c91-4bea-9811-4535ae9f00ef-metrics-certs\") pod \"maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg\" (UID: \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:30:58.739010 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.738902    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d61e92-8c91-4bea-9811-4535ae9f00ef-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg\" (UID: \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:30:58.739010 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.738933    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdnzt\" (UniqueName: \"kubernetes.io/projected/c4d61e92-8c91-4bea-9811-4535ae9f00ef-kube-api-access-hdnzt\") pod \"maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg\" (UID: \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:30:58.739123 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.739035    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4d61e92-8c91-4bea-9811-4535ae9f00ef-maas-api-tls\") pod \"maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg\" (UID: \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:30:58.739213 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:30:58.739192    4521 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-serving-cert: secret "maas-api-e2e-worker-w3-82c724-serving-cert" not found
Nov 04 03:30:58.739276 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:30:58.739266    4521 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4d61e92-8c91-4bea-9811-4535ae9f00ef-maas-api-tls podName:c4d61e92-8c91-4bea-9811-4535ae9f00ef nodeName:}" failed. No retries permitted until 2026-11-04 03:30:59.23924525 +0000 UTC m=+825.443845557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c4d61e92-8c91-4bea-9811-4535ae9f00ef-maas-api-tls") pod "maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg" (UID: "c4d61e92-8c91-4bea-9811-4535ae9f00ef") : secret "maas-api-e2e-worker-w3-82c724-serving-cert" not found
Nov 04 03:30:58.739973 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.739952    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d61e92-8c91-4bea-9811-4535ae9f00ef-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg\" (UID: \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:30:58.741385 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.741363    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4d61e92-8c91-4bea-9811-4535ae9f00ef-metrics-certs\") pod \"maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg\" (UID: \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:30:58.800510 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:58.800483    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdnzt\" (UniqueName: \"kubernetes.io/projected/c4d61e92-8c91-4bea-9811-4535ae9f00ef-kube-api-access-hdnzt\") pod \"maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg\" (UID: \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:30:59.244469 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:59.244431    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4d61e92-8c91-4bea-9811-4535ae9f00ef-maas-api-tls\") pod \"maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg\" (UID: \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:30:59.246899 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:59.246838    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4d61e92-8c91-4bea-9811-4535ae9f00ef-maas-api-tls\") pod \"maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg\" (UID: \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:30:59.352306 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:59.352266    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh" event={"ID":"78ed61dd-e0d5-4165-ad93-58d761dfdd00","Type":"ContainerStarted","Data":"82fe11f0362c0dd8c7e74ceeff0317b3ab313d309d1a831666bd0021f758de0f"}
Nov 04 03:30:59.352449 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:59.352364    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh" event={"ID":"78ed61dd-e0d5-4165-ad93-58d761dfdd00","Type":"ContainerStarted","Data":"61a1e77260be0ae7f1103f6799e9f8ddff6d55781d3ebabab2df28c15fe37aca"}
Nov 04 03:30:59.475080 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:59.475029    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh" podStartSLOduration=2.475013956 podStartE2EDuration="2.475013956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:30:59.473233943 +0000 UTC m=+825.677834275" watchObservedRunningTime="2026-11-04 03:30:59.475013956 +0000 UTC m=+825.679614258"
Nov 04 03:30:59.479430 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:59.479403    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:30:59.853762 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:30:59.853608    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"]
Nov 04 03:30:59.857803 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:30:59.857765    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d61e92_8c91_4bea_9811_4535ae9f00ef.slice/crio-7ad10456dd2389876e6650440df6b7c3e5b64a15d71974990a6968fe12b76077 WatchSource:0}: Error finding container 7ad10456dd2389876e6650440df6b7c3e5b64a15d71974990a6968fe12b76077: Status 404 returned error can't find the container with id 7ad10456dd2389876e6650440df6b7c3e5b64a15d71974990a6968fe12b76077
Nov 04 03:31:00.150206 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:00.150121    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:31:00.156691 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:00.156664    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:31:00.358358 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:00.358326    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg" event={"ID":"c4d61e92-8c91-4bea-9811-4535ae9f00ef","Type":"ContainerStarted","Data":"7ad10456dd2389876e6650440df6b7c3e5b64a15d71974990a6968fe12b76077"}
Nov 04 03:31:00.358585 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:00.358564    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:31:00.359578 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:00.359560    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:31:01.365092 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:01.365054    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg" event={"ID":"c4d61e92-8c91-4bea-9811-4535ae9f00ef","Type":"ContainerStarted","Data":"5ee64d88b2f534cdc8a134f7a5fa3561f969be46479c6e4cc834ee5486eed425"}
Nov 04 03:31:01.365598 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:01.365382    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:31:01.491416 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:01.491359    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg" podStartSLOduration=2.581447469 podStartE2EDuration="3.491341353s" totalImagesPullingTime="909.893884ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:31:01.490587038 +0000 UTC m=+827.695187341" watchObservedRunningTime="2026-11-04 03:31:01.491341353 +0000 UTC m=+827.695941656"
Nov 04 03:31:03.882490 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:03.882459    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"]
Nov 04 03:31:03.886324 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:03.886304    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:31:03.939362 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:03.939331    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-fba151-serving-cert\""
Nov 04 03:31:03.991565 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:03.991534    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"]
Nov 04 03:31:03.993159 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:03.993139    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c6d1176-f442-4f20-84e5-f89e0c12f263-metrics-certs\") pod \"maas-api-e2e-worker-w2-fba151-746847b655-dxf47\" (UID: \"3c6d1176-f442-4f20-84e5-f89e0c12f263\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:31:03.993212 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:03.993197    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c6d1176-f442-4f20-84e5-f89e0c12f263-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-fba151-746847b655-dxf47\" (UID: \"3c6d1176-f442-4f20-84e5-f89e0c12f263\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:31:03.993373 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:03.993361    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7qzp\" (UniqueName: \"kubernetes.io/projected/3c6d1176-f442-4f20-84e5-f89e0c12f263-kube-api-access-m7qzp\") pod \"maas-api-e2e-worker-w2-fba151-746847b655-dxf47\" (UID: \"3c6d1176-f442-4f20-84e5-f89e0c12f263\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:31:03.993468 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:03.993451    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c6d1176-f442-4f20-84e5-f89e0c12f263-maas-api-tls\") pod \"maas-api-e2e-worker-w2-fba151-746847b655-dxf47\" (UID: \"3c6d1176-f442-4f20-84e5-f89e0c12f263\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:31:04.094996 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:04.094964    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c6d1176-f442-4f20-84e5-f89e0c12f263-metrics-certs\") pod \"maas-api-e2e-worker-w2-fba151-746847b655-dxf47\" (UID: \"3c6d1176-f442-4f20-84e5-f89e0c12f263\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:31:04.095193 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:04.095025    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c6d1176-f442-4f20-84e5-f89e0c12f263-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-fba151-746847b655-dxf47\" (UID: \"3c6d1176-f442-4f20-84e5-f89e0c12f263\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:31:04.095193 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:04.095062    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7qzp\" (UniqueName: \"kubernetes.io/projected/3c6d1176-f442-4f20-84e5-f89e0c12f263-kube-api-access-m7qzp\") pod \"maas-api-e2e-worker-w2-fba151-746847b655-dxf47\" (UID: \"3c6d1176-f442-4f20-84e5-f89e0c12f263\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:31:04.095193 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:04.095084    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c6d1176-f442-4f20-84e5-f89e0c12f263-maas-api-tls\") pod \"maas-api-e2e-worker-w2-fba151-746847b655-dxf47\" (UID: \"3c6d1176-f442-4f20-84e5-f89e0c12f263\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:31:04.095370 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:31:04.095220    4521 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-serving-cert: secret "maas-api-e2e-worker-w2-fba151-serving-cert" not found
Nov 04 03:31:04.095370 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:31:04.095280    4521 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c6d1176-f442-4f20-84e5-f89e0c12f263-maas-api-tls podName:3c6d1176-f442-4f20-84e5-f89e0c12f263 nodeName:}" failed. No retries permitted until 2026-11-04 03:31:04.595259166 +0000 UTC m=+830.799859460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3c6d1176-f442-4f20-84e5-f89e0c12f263-maas-api-tls") pod "maas-api-e2e-worker-w2-fba151-746847b655-dxf47" (UID: "3c6d1176-f442-4f20-84e5-f89e0c12f263") : secret "maas-api-e2e-worker-w2-fba151-serving-cert" not found
Nov 04 03:31:04.095759 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:04.095737    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c6d1176-f442-4f20-84e5-f89e0c12f263-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-fba151-746847b655-dxf47\" (UID: \"3c6d1176-f442-4f20-84e5-f89e0c12f263\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:31:04.097433 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:04.097413    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c6d1176-f442-4f20-84e5-f89e0c12f263-metrics-certs\") pod \"maas-api-e2e-worker-w2-fba151-746847b655-dxf47\" (UID: \"3c6d1176-f442-4f20-84e5-f89e0c12f263\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:31:04.159258 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:04.159185    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7qzp\" (UniqueName: \"kubernetes.io/projected/3c6d1176-f442-4f20-84e5-f89e0c12f263-kube-api-access-m7qzp\") pod \"maas-api-e2e-worker-w2-fba151-746847b655-dxf47\" (UID: \"3c6d1176-f442-4f20-84e5-f89e0c12f263\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:31:04.601160 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:04.601122    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c6d1176-f442-4f20-84e5-f89e0c12f263-maas-api-tls\") pod \"maas-api-e2e-worker-w2-fba151-746847b655-dxf47\" (UID: \"3c6d1176-f442-4f20-84e5-f89e0c12f263\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:31:04.603504 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:04.603478    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c6d1176-f442-4f20-84e5-f89e0c12f263-maas-api-tls\") pod \"maas-api-e2e-worker-w2-fba151-746847b655-dxf47\" (UID: \"3c6d1176-f442-4f20-84e5-f89e0c12f263\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:31:04.800067 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:04.800035    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:31:05.177002 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:05.176980    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"]
Nov 04 03:31:05.179160 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:31:05.179131    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c6d1176_f442_4f20_84e5_f89e0c12f263.slice/crio-6c74f5ae841059374814918dbbea1fe6e5ed4301e746c5d6b9ac6161b0db90c5 WatchSource:0}: Error finding container 6c74f5ae841059374814918dbbea1fe6e5ed4301e746c5d6b9ac6161b0db90c5: Status 404 returned error can't find the container with id 6c74f5ae841059374814918dbbea1fe6e5ed4301e746c5d6b9ac6161b0db90c5
Nov 04 03:31:05.387319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:05.387282    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47" event={"ID":"3c6d1176-f442-4f20-84e5-f89e0c12f263","Type":"ContainerStarted","Data":"6c74f5ae841059374814918dbbea1fe6e5ed4301e746c5d6b9ac6161b0db90c5"}
Nov 04 03:31:06.393970 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:06.393874    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47" event={"ID":"3c6d1176-f442-4f20-84e5-f89e0c12f263","Type":"ContainerStarted","Data":"3ff5440858dc62872a4a2f511ab019a15343013aac9d20b9978662654fcc1b66"}
Nov 04 03:31:06.394361 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:06.394001    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:31:06.513564 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:06.513508    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47" podStartSLOduration=2.591839748 podStartE2EDuration="3.513470603s" totalImagesPullingTime="921.630855ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:31:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:31:06.512616666 +0000 UTC m=+832.717217014" watchObservedRunningTime="2026-11-04 03:31:06.513470603 +0000 UTC m=+832.718070959"
Nov 04 03:31:07.383685 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:07.383655    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:31:08.014611 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:08.014576    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"]
Nov 04 03:31:08.018323 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:08.018305    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"
Nov 04 03:31:08.071127 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:08.071100    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-ee1947\""
Nov 04 03:31:08.071334 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:08.071317    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-ee1947-dockercfg-m78fb\""
Nov 04 03:31:08.076828 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:08.076802    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"]
Nov 04 03:31:08.135530 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:08.135493    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdjqm\" (UniqueName: \"kubernetes.io/projected/1b7602b8-2d7b-42b4-acd3-868233206892-kube-api-access-cdjqm\") pod \"payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq\" (UID: \"1b7602b8-2d7b-42b4-acd3-868233206892\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"
Nov 04 03:31:08.135727 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:08.135571    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7602b8-2d7b-42b4-acd3-868233206892-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq\" (UID: \"1b7602b8-2d7b-42b4-acd3-868233206892\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"
Nov 04 03:31:08.237140 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:08.237102    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdjqm\" (UniqueName: \"kubernetes.io/projected/1b7602b8-2d7b-42b4-acd3-868233206892-kube-api-access-cdjqm\") pod \"payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq\" (UID: \"1b7602b8-2d7b-42b4-acd3-868233206892\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"
Nov 04 03:31:08.237330 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:08.237164    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7602b8-2d7b-42b4-acd3-868233206892-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq\" (UID: \"1b7602b8-2d7b-42b4-acd3-868233206892\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"
Nov 04 03:31:08.238028 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:08.237994    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7602b8-2d7b-42b4-acd3-868233206892-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq\" (UID: \"1b7602b8-2d7b-42b4-acd3-868233206892\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"
Nov 04 03:31:08.298833 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:08.298802    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdjqm\" (UniqueName: \"kubernetes.io/projected/1b7602b8-2d7b-42b4-acd3-868233206892-kube-api-access-cdjqm\") pod \"payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq\" (UID: \"1b7602b8-2d7b-42b4-acd3-868233206892\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"
Nov 04 03:31:08.331676 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:08.331642    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"
Nov 04 03:31:08.348482 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:08.348461    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"
Nov 04 03:31:08.348631 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:08.348526    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"
Nov 04 03:31:08.706056 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:08.706029    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"]
Nov 04 03:31:08.707936 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:31:08.707901    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b7602b8_2d7b_42b4_acd3_868233206892.slice/crio-76afaf7d94e7f37e3817d1a9f7b41bd76b0a139f96ffa6b4155b5635b32e518a WatchSource:0}: Error finding container 76afaf7d94e7f37e3817d1a9f7b41bd76b0a139f96ffa6b4155b5635b32e518a: Status 404 returned error can't find the container with id 76afaf7d94e7f37e3817d1a9f7b41bd76b0a139f96ffa6b4155b5635b32e518a
Nov 04 03:31:09.409884 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:09.409822    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq" event={"ID":"1b7602b8-2d7b-42b4-acd3-868233206892","Type":"ContainerStarted","Data":"6c97d4c14e24737ac15ca296dcd21fc061e0ec1c653e4ef8f99048d8436655a3"}
Nov 04 03:31:09.410332 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:09.409908    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq" event={"ID":"1b7602b8-2d7b-42b4-acd3-868233206892","Type":"ContainerStarted","Data":"76afaf7d94e7f37e3817d1a9f7b41bd76b0a139f96ffa6b4155b5635b32e518a"}
Nov 04 03:31:09.410332 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:09.410022    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"
Nov 04 03:31:09.527450 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:09.527393    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq" podStartSLOduration=2.527373865 podStartE2EDuration="2.527373865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:31:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:31:09.526216955 +0000 UTC m=+835.730817258" watchObservedRunningTime="2026-11-04 03:31:09.527373865 +0000 UTC m=+835.731974170"
Nov 04 03:31:10.571023 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:10.570984    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"]
Nov 04 03:31:10.574663 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:10.574646    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"
Nov 04 03:31:10.627718 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:10.627692    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-50429e-dockercfg-qlgdz\""
Nov 04 03:31:10.663270 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:10.663221    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58m7z\" (UniqueName: \"kubernetes.io/projected/c8e3aedf-25cb-4002-8fd3-70b762747e3e-kube-api-access-58m7z\") pod \"payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx\" (UID: \"c8e3aedf-25cb-4002-8fd3-70b762747e3e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"
Nov 04 03:31:10.663488 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:10.663473    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8e3aedf-25cb-4002-8fd3-70b762747e3e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx\" (UID: \"c8e3aedf-25cb-4002-8fd3-70b762747e3e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"
Nov 04 03:31:10.678105 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:10.678065    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-50429e\""
Nov 04 03:31:10.680081 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:10.680062    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"]
Nov 04 03:31:10.764328 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:10.764292    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8e3aedf-25cb-4002-8fd3-70b762747e3e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx\" (UID: \"c8e3aedf-25cb-4002-8fd3-70b762747e3e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"
Nov 04 03:31:10.764492 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:10.764377    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58m7z\" (UniqueName: \"kubernetes.io/projected/c8e3aedf-25cb-4002-8fd3-70b762747e3e-kube-api-access-58m7z\") pod \"payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx\" (UID: \"c8e3aedf-25cb-4002-8fd3-70b762747e3e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"
Nov 04 03:31:10.765093 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:10.765071    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8e3aedf-25cb-4002-8fd3-70b762747e3e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx\" (UID: \"c8e3aedf-25cb-4002-8fd3-70b762747e3e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"
Nov 04 03:31:10.825681 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:10.825615    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58m7z\" (UniqueName: \"kubernetes.io/projected/c8e3aedf-25cb-4002-8fd3-70b762747e3e-kube-api-access-58m7z\") pod \"payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx\" (UID: \"c8e3aedf-25cb-4002-8fd3-70b762747e3e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"
Nov 04 03:31:10.896361 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:10.896335    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"
Nov 04 03:31:11.270175 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:11.270149    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"]
Nov 04 03:31:11.272743 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:31:11.272711    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e3aedf_25cb_4002_8fd3_70b762747e3e.slice/crio-a1fd41d50084b9c366390515a3e8892b76620a34acba01b6eaa35f815a75394c WatchSource:0}: Error finding container a1fd41d50084b9c366390515a3e8892b76620a34acba01b6eaa35f815a75394c: Status 404 returned error can't find the container with id a1fd41d50084b9c366390515a3e8892b76620a34acba01b6eaa35f815a75394c
Nov 04 03:31:11.421963 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:11.421927    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx" event={"ID":"c8e3aedf-25cb-4002-8fd3-70b762747e3e","Type":"ContainerStarted","Data":"772681783b2eedbc2486ed64096150bf00981774035300a17a0507db9d5de48f"}
Nov 04 03:31:11.422119 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:11.421981    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx" event={"ID":"c8e3aedf-25cb-4002-8fd3-70b762747e3e","Type":"ContainerStarted","Data":"a1fd41d50084b9c366390515a3e8892b76620a34acba01b6eaa35f815a75394c"}
Nov 04 03:31:11.422119 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:11.422070    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"
Nov 04 03:31:11.538706 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:11.538537    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx" podStartSLOduration=1.538520122 podStartE2EDuration="1.538520122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:31:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:31:11.538059181 +0000 UTC m=+837.742659482" watchObservedRunningTime="2026-11-04 03:31:11.538520122 +0000 UTC m=+837.743120424"
Nov 04 03:31:12.403664 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:12.403635    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:31:14.407643 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:14.407613    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"]
Nov 04 03:31:14.410976 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:14.410958    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:31:14.463864 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:14.463820    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-ee1947-serving-cert\""
Nov 04 03:31:14.501154 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:14.501116    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt\" (UID: \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:31:14.501319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:14.501169    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-maas-api-tls\") pod \"maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt\" (UID: \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:31:14.501319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:14.501272    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-metrics-certs\") pod \"maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt\" (UID: \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:31:14.501319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:14.501297    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcd8r\" (UniqueName: \"kubernetes.io/projected/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-kube-api-access-jcd8r\") pod \"maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt\" (UID: \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:31:14.516165 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:14.516142    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"]
Nov 04 03:31:14.602970 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:14.602930    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-maas-api-tls\") pod \"maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt\" (UID: \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:31:14.603156 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:14.602997    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-metrics-certs\") pod \"maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt\" (UID: \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:31:14.603156 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:14.603016    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcd8r\" (UniqueName: \"kubernetes.io/projected/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-kube-api-access-jcd8r\") pod \"maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt\" (UID: \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:31:14.603156 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:31:14.603100    4521 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-serving-cert: secret "maas-api-e2e-worker-w1-ee1947-serving-cert" not found
Nov 04 03:31:14.603156 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:14.603134    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt\" (UID: \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:31:14.603358 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:31:14.603177    4521 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-maas-api-tls podName:3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7 nodeName:}" failed. No retries permitted until 2026-11-04 03:31:15.103153525 +0000 UTC m=+841.307753810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-maas-api-tls") pod "maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt" (UID: "3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7") : secret "maas-api-e2e-worker-w1-ee1947-serving-cert" not found
Nov 04 03:31:14.603746 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:14.603728    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt\" (UID: \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:31:14.605304 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:14.605280    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-metrics-certs\") pod \"maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt\" (UID: \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:31:14.665888 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:14.665788    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcd8r\" (UniqueName: \"kubernetes.io/projected/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-kube-api-access-jcd8r\") pod \"maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt\" (UID: \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:31:15.108952 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:15.108913    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-maas-api-tls\") pod \"maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt\" (UID: \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:31:15.111489 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:15.111457    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-maas-api-tls\") pod \"maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt\" (UID: \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:31:15.323350 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:15.323302    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:31:15.698773 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:15.698747    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"]
Nov 04 03:31:15.700793 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:31:15.700765    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c67035d_6e80_4b8b_9c0e_5dc3a1b4b4a7.slice/crio-d037ea2a29df03a7b7f99e58afb045bf690428d8e4be49316d188242e60fd504 WatchSource:0}: Error finding container d037ea2a29df03a7b7f99e58afb045bf690428d8e4be49316d188242e60fd504: Status 404 returned error can't find the container with id d037ea2a29df03a7b7f99e58afb045bf690428d8e4be49316d188242e60fd504
Nov 04 03:31:16.448121 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:16.448087    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt" event={"ID":"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7","Type":"ContainerStarted","Data":"d037ea2a29df03a7b7f99e58afb045bf690428d8e4be49316d188242e60fd504"}
Nov 04 03:31:17.454638 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:17.454597    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt" event={"ID":"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7","Type":"ContainerStarted","Data":"afe5190f0a335a1d6efcffd5e1a01bd03818dddb50880ff9e3247339bb8dea0a"}
Nov 04 03:31:17.455048 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:17.454666    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:31:17.624038 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:17.623980    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt" podStartSLOduration=2.7255257520000002 podStartE2EDuration="3.623959758s" totalImagesPullingTime="898.434006ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:31:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:31:17.572869385 +0000 UTC m=+843.777469682" watchObservedRunningTime="2026-11-04 03:31:17.623959758 +0000 UTC m=+843.828560063"
Nov 04 03:31:20.418701 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:20.418669    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"
Nov 04 03:31:22.431279 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:22.431249    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"
Nov 04 03:31:23.465319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:23.465293    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:31:25.646808 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:25.646776    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"]
Nov 04 03:31:25.650628 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:25.650612    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:31:25.703720 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:25.703695    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-50429e-serving-cert\""
Nov 04 03:31:25.756215 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:25.756185    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"]
Nov 04 03:31:25.817635 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:25.817588    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p\" (UID: \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:31:25.817804 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:25.817718    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r5jl\" (UniqueName: \"kubernetes.io/projected/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-kube-api-access-2r5jl\") pod \"maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p\" (UID: \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:31:25.817804 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:25.817787    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-maas-api-tls\") pod \"maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p\" (UID: \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:31:25.818078 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:25.818047    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-metrics-certs\") pod \"maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p\" (UID: \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:31:25.918935 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:25.918831    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-metrics-certs\") pod \"maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p\" (UID: \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:31:25.918935 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:25.918885    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p\" (UID: \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:31:25.918935 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:25.918914    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r5jl\" (UniqueName: \"kubernetes.io/projected/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-kube-api-access-2r5jl\") pod \"maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p\" (UID: \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:31:25.919172 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:25.918943    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-maas-api-tls\") pod \"maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p\" (UID: \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:31:25.919172 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:31:25.919057    4521 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-serving-cert: secret "maas-api-e2e-worker-w4-50429e-serving-cert" not found
Nov 04 03:31:25.919172 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:31:25.919114    4521 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-maas-api-tls podName:3cad7a57-df1d-4c61-ad35-5c8ddbaecc00 nodeName:}" failed. No retries permitted until 2026-11-04 03:31:26.41910014 +0000 UTC m=+852.623700434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-maas-api-tls") pod "maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p" (UID: "3cad7a57-df1d-4c61-ad35-5c8ddbaecc00") : secret "maas-api-e2e-worker-w4-50429e-serving-cert" not found
Nov 04 03:31:25.919607 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:25.919588    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p\" (UID: \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:31:25.921191 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:25.921173    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-metrics-certs\") pod \"maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p\" (UID: \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:31:25.981255 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:25.981224    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r5jl\" (UniqueName: \"kubernetes.io/projected/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-kube-api-access-2r5jl\") pod \"maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p\" (UID: \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:31:26.424092 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:26.424051    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-maas-api-tls\") pod \"maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p\" (UID: \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:31:26.426525 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:26.426502    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-maas-api-tls\") pod \"maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p\" (UID: \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:31:26.591478 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:26.591443    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:31:26.965097 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:26.965072    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"]
Nov 04 03:31:26.966807 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:31:26.966780    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cad7a57_df1d_4c61_ad35_5c8ddbaecc00.slice/crio-f58f29363ed9b5f0fb4ed283a2a9e5ba5956864d9adc982b34a29d80bffba989 WatchSource:0}: Error finding container f58f29363ed9b5f0fb4ed283a2a9e5ba5956864d9adc982b34a29d80bffba989: Status 404 returned error can't find the container with id f58f29363ed9b5f0fb4ed283a2a9e5ba5956864d9adc982b34a29d80bffba989
Nov 04 03:31:27.502748 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:27.502710    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p" event={"ID":"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00","Type":"ContainerStarted","Data":"f58f29363ed9b5f0fb4ed283a2a9e5ba5956864d9adc982b34a29d80bffba989"}
Nov 04 03:31:28.509026 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:28.508994    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p" event={"ID":"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00","Type":"ContainerStarted","Data":"38f3e4cd31cbc3e9b2d9be7c282f76abe88d7a7d550f8b911d2f48d7ee0ece4c"}
Nov 04 03:31:28.509386 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:28.509108    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:31:28.630244 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:28.630185    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p" podStartSLOduration=2.726113899 podStartE2EDuration="3.630171417s" totalImagesPullingTime="904.057518ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:31:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:31:28.628445386 +0000 UTC m=+854.833045687" watchObservedRunningTime="2026-11-04 03:31:28.630171417 +0000 UTC m=+854.834771761"
Nov 04 03:31:28.731025 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:31:28.730990    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:31:34.519474 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:34.519444    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:31:41.265264 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:41.265215    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"]
Nov 04 03:31:41.270417 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:41.270393    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:31:41.294948 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:41.294907    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2\" (UID: \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:31:41.295152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:41.295105    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-maas-api-tls\") pod \"maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2\" (UID: \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:31:41.295152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:41.295138    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mfbf\" (UniqueName: \"kubernetes.io/projected/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-kube-api-access-9mfbf\") pod \"maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2\" (UID: \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:31:41.295261 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:41.295235    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-metrics-certs\") pod \"maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2\" (UID: \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:31:41.326751 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:41.326721    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-092f8c-serving-cert\""
Nov 04 03:31:41.377984 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:41.377943    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"]
Nov 04 03:31:41.396666 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:41.396628    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-maas-api-tls\") pod \"maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2\" (UID: \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:31:41.396878 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:41.396678    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mfbf\" (UniqueName: \"kubernetes.io/projected/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-kube-api-access-9mfbf\") pod \"maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2\" (UID: \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:31:41.396878 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:41.396757    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-metrics-certs\") pod \"maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2\" (UID: \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:31:41.397020 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:41.396886    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2\" (UID: \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:31:41.397214 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:31:41.397186    4521 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-serving-cert: secret "maas-api-e2e-worker-w0-092f8c-serving-cert" not found
Nov 04 03:31:41.397311 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:31:41.397279    4521 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-maas-api-tls podName:82634af9-adbd-4085-9f4b-a4ab21d7b2d3 nodeName:}" failed. No retries permitted until 2026-11-04 03:31:41.897255976 +0000 UTC m=+868.101856278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-maas-api-tls") pod "maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2" (UID: "82634af9-adbd-4085-9f4b-a4ab21d7b2d3") : secret "maas-api-e2e-worker-w0-092f8c-serving-cert" not found
Nov 04 03:31:41.397774 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:41.397740    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2\" (UID: \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:31:41.402485 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:41.402455    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-metrics-certs\") pod \"maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2\" (UID: \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:31:41.458783 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:41.458747    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mfbf\" (UniqueName: \"kubernetes.io/projected/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-kube-api-access-9mfbf\") pod \"maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2\" (UID: \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:31:41.902543 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:41.902511    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-maas-api-tls\") pod \"maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2\" (UID: \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:31:41.905372 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:41.905340    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-maas-api-tls\") pod \"maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2\" (UID: \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:31:42.184643 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:42.184579    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:31:42.561268 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:42.561220    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"]
Nov 04 03:31:42.564120 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:31:42.564086    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82634af9_adbd_4085_9f4b_a4ab21d7b2d3.slice/crio-a5612fdf5c296804714db08d4d2cbc1fa259f7978688dfa9e8dca061da05836e WatchSource:0}: Error finding container a5612fdf5c296804714db08d4d2cbc1fa259f7978688dfa9e8dca061da05836e: Status 404 returned error can't find the container with id a5612fdf5c296804714db08d4d2cbc1fa259f7978688dfa9e8dca061da05836e
Nov 04 03:31:42.582979 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:42.582949    4521 generic.go:356] "Generic (PLEG): container finished" podID="78ed61dd-e0d5-4165-ad93-58d761dfdd00" containerID="82fe11f0362c0dd8c7e74ceeff0317b3ab313d309d1a831666bd0021f758de0f" exitCode=0
Nov 04 03:31:42.583105 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:42.583023    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh" event={"ID":"78ed61dd-e0d5-4165-ad93-58d761dfdd00","Type":"ContainerDied","Data":"82fe11f0362c0dd8c7e74ceeff0317b3ab313d309d1a831666bd0021f758de0f"}
Nov 04 03:31:42.583477 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:42.583443    4521 scope.go:122] "RemoveContainer" containerID="82fe11f0362c0dd8c7e74ceeff0317b3ab313d309d1a831666bd0021f758de0f"
Nov 04 03:31:42.584312 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:42.584289    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2" event={"ID":"82634af9-adbd-4085-9f4b-a4ab21d7b2d3","Type":"ContainerStarted","Data":"a5612fdf5c296804714db08d4d2cbc1fa259f7978688dfa9e8dca061da05836e"}
Nov 04 03:31:42.584786 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:42.584758    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:31:42.584838 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:42.584826    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:31:42.584899 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:42.584885    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:31:43.617220 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:43.617180    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh" event={"ID":"78ed61dd-e0d5-4165-ad93-58d761dfdd00","Type":"ContainerStarted","Data":"099a4693d988840f36bb62fa3381a283559edbf745a5504a1f829b3a626cf51e"}
Nov 04 03:31:43.618967 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:43.618933    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2" event={"ID":"82634af9-adbd-4085-9f4b-a4ab21d7b2d3","Type":"ContainerStarted","Data":"3e7ebea143df986c9771b01444d1e79b355891827b6b48a8a6bb9ab430084bf5"}
Nov 04 03:31:43.619114 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:43.618995    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:31:43.872915 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:43.872858    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2" podStartSLOduration=1.92124197 podStartE2EDuration="2.872826971s" totalImagesPullingTime="951.585001ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:31:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:31:43.871313661 +0000 UTC m=+870.075913964" watchObservedRunningTime="2026-11-04 03:31:43.872826971 +0000 UTC m=+870.077427273"
Nov 04 03:31:44.150194 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:44.150116    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:31:44.150943 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:44.150918    4521 patch_prober.go:29] interesting pod/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Nov 04 03:31:44.151062 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:44.150972    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh" podUID="78ed61dd-e0d5-4165-ad93-58d761dfdd00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Nov 04 03:31:45.149961 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:45.149928    4521 patch_prober.go:29] interesting pod/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Nov 04 03:31:45.150346 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:45.149977    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh" podUID="78ed61dd-e0d5-4165-ad93-58d761dfdd00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Nov 04 03:31:45.359349 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:45.359311    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:31:46.150518 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:46.150473    4521 patch_prober.go:29] interesting pod/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Nov 04 03:31:46.150947 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:46.150550    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh" podUID="78ed61dd-e0d5-4165-ad93-58d761dfdd00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Nov 04 03:31:47.150140 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:47.150109    4521 patch_prober.go:29] interesting pod/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Nov 04 03:31:47.150331 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:47.150162    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh" podUID="78ed61dd-e0d5-4165-ad93-58d761dfdd00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Nov 04 03:31:48.150516 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:48.150481    4521 patch_prober.go:29] interesting pod/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Nov 04 03:31:48.150950 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:48.150529    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh" podUID="78ed61dd-e0d5-4165-ad93-58d761dfdd00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Nov 04 03:31:49.154520 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:49.154491    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:31:49.155465 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:49.155443    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:31:49.630422 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:31:49.630395    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:31:58.730953 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:31:58.730834    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:32:11.062209 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.062172    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"]
Nov 04 03:32:11.065328 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.065309    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.125246 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.125200    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbmgt\" (UniqueName: \"kubernetes.io/projected/c53728db-b02a-497b-9a90-7f47a026e656-kube-api-access-tbmgt\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.125246 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.125247    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-model-cache\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.125453 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.125312    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-dshm\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.125453 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.125352    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-tmp-dir\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.125453 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.125380    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c53728db-b02a-497b-9a90-7f47a026e656-tls-certs\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.125453 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.125401    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-home\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.168851 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.168824    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-82c724\"/\"openshift-service-ca.crt\""
Nov 04 03:32:11.169020 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.168892    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-82c724\"/\"kube-root-ca.crt\""
Nov 04 03:32:11.169020 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.169007    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-82c724\"/\"premium-w3-82c724-kserve-self-signed-certs\""
Nov 04 03:32:11.169121 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.169015    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-82c724\"/\"default-dockercfg-rsnz8\""
Nov 04 03:32:11.171248 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.171225    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"]
Nov 04 03:32:11.227009 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.226975    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbmgt\" (UniqueName: \"kubernetes.io/projected/c53728db-b02a-497b-9a90-7f47a026e656-kube-api-access-tbmgt\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.227187 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.227015    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-model-cache\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.227187 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.227162    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-dshm\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.227302 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.227197    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-tmp-dir\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.227302 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.227226    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c53728db-b02a-497b-9a90-7f47a026e656-tls-certs\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.227302 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.227269    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-home\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.227585 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.227337    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-model-cache\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.227585 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.227561    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-tmp-dir\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.227656 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.227615    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-home\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.229376 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.229350    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-dshm\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.229764 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.229697    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c53728db-b02a-497b-9a90-7f47a026e656-tls-certs\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.288972 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.288944    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbmgt\" (UniqueName: \"kubernetes.io/projected/c53728db-b02a-497b-9a90-7f47a026e656-kube-api-access-tbmgt\") pod \"premium-w3-82c724-kserve-9c894d4fc-gf8jv\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") " pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.376628 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.376548    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:11.751941 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.751917    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"]
Nov 04 03:32:11.755080 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:32:11.755050    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc53728db_b02a_497b_9a90_7f47a026e656.slice/crio-8dc48279e6c2d296be0399b10618e7bb246b0e886102272b6705b5936137a0b3 WatchSource:0}: Error finding container 8dc48279e6c2d296be0399b10618e7bb246b0e886102272b6705b5936137a0b3: Status 404 returned error can't find the container with id 8dc48279e6c2d296be0399b10618e7bb246b0e886102272b6705b5936137a0b3
Nov 04 03:32:11.770504 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:11.770478    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv" event={"ID":"c53728db-b02a-497b-9a90-7f47a026e656","Type":"ContainerStarted","Data":"8dc48279e6c2d296be0399b10618e7bb246b0e886102272b6705b5936137a0b3"}
Nov 04 03:32:12.777341 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:12.777301    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv" event={"ID":"c53728db-b02a-497b-9a90-7f47a026e656","Type":"ContainerStarted","Data":"dc8e5c1527037e5c8860e31cb626bf5fa1362517adfdaa0401f05b973f06a29b"}
Nov 04 03:32:12.897563 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:12.897512    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv" podStartSLOduration=1.897499475 podStartE2EDuration="1.897499475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:32:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:32:12.895002923 +0000 UTC m=+899.099603252" watchObservedRunningTime="2026-11-04 03:32:12.897499475 +0000 UTC m=+899.102099798"
Nov 04 03:32:13.377280 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:13.377245    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:21.377375 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:21.377339    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:21.390102 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:21.390074    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:21.843114 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:21.843085    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:32:28.730280 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:32:28.730241    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:32:37.669183 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.669150    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"]
Nov 04 03:32:37.672245 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.672228    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.729057 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.729025    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-50429e\"/\"openshift-service-ca.crt\""
Nov 04 03:32:37.775931 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.775896    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-50429e\"/\"default-dockercfg-q5mmj\""
Nov 04 03:32:37.776088 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.775908    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-50429e\"/\"kube-root-ca.crt\""
Nov 04 03:32:37.776088 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.775957    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-50429e\"/\"simulator-w4-50429e-kserve-self-signed-certs\""
Nov 04 03:32:37.778218 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.778197    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"]
Nov 04 03:32:37.818722 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.818688    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-dshm\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.819042 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.819010    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-tmp-dir\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.819251 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.819068    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-model-cache\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.819251 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.819143    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcbm4\" (UniqueName: \"kubernetes.io/projected/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-kube-api-access-zcbm4\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.819404 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.819319    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-tls-certs\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.819404 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.819371    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-home\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.920363 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.920288    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-tmp-dir\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.920363 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.920325    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-model-cache\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.920363 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.920343    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcbm4\" (UniqueName: \"kubernetes.io/projected/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-kube-api-access-zcbm4\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.920651 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.920405    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-tls-certs\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.920651 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.920424    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-home\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.920651 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.920458    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-dshm\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.920821 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.920729    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-tmp-dir\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.920821 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.920781    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-model-cache\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.920960 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.920872    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-home\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.922797 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.922772    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-dshm\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.923061 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.923044    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-tls-certs\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.982070 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.982041    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcbm4\" (UniqueName: \"kubernetes.io/projected/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-kube-api-access-zcbm4\") pod \"simulator-w4-50429e-kserve-67d765f84-bgkh9\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") " pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:37.983943 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:37.983923    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:38.357092 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:38.357072    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"]
Nov 04 03:32:38.359160 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:32:38.359136    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69b690e9_f65d_42c9_9ca3_aea2ec55f30a.slice/crio-5723e3668d3a9c5b665881dfccd5b8c565272b41768259c2a1155f081d6bda4a WatchSource:0}: Error finding container 5723e3668d3a9c5b665881dfccd5b8c565272b41768259c2a1155f081d6bda4a: Status 404 returned error can't find the container with id 5723e3668d3a9c5b665881dfccd5b8c565272b41768259c2a1155f081d6bda4a
Nov 04 03:32:38.919927 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:38.919891    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9" event={"ID":"69b690e9-f65d-42c9-9ca3-aea2ec55f30a","Type":"ContainerStarted","Data":"fa84ef26809e229ef196c3c3dfe502d0452e92f040ec7b2189350095d4a78533"}
Nov 04 03:32:38.919927 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:38.919926    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9" event={"ID":"69b690e9-f65d-42c9-9ca3-aea2ec55f30a","Type":"ContainerStarted","Data":"5723e3668d3a9c5b665881dfccd5b8c565272b41768259c2a1155f081d6bda4a"}
Nov 04 03:32:38.984361 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:38.984334    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:39.037065 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:39.037003    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9" podStartSLOduration=2.036987212 podStartE2EDuration="2.036987212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:32:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:32:39.036952031 +0000 UTC m=+925.241552334" watchObservedRunningTime="2026-11-04 03:32:39.036987212 +0000 UTC m=+925.241587516"
Nov 04 03:32:47.984593 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:47.984549    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:47.996989 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:47.996965    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:48.986625 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:32:48.986595    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:32:58.729190 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:32:58.729100    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:33:22.193355 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.193321    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"]
Nov 04 03:33:22.197616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.197599    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.271664 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.271632    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-ee1947\"/\"kube-root-ca.crt\""
Nov 04 03:33:22.291566 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.291536    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-dshm\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.291703 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.291662    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-home\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.291754 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.291735    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3eb20c6-1760-4310-9edf-7b6392b89b63-tls-certs\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.291798 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.291764    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-tmp-dir\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.291861 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.291829    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zjf8\" (UniqueName: \"kubernetes.io/projected/b3eb20c6-1760-4310-9edf-7b6392b89b63-kube-api-access-2zjf8\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.291916 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.291903    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-model-cache\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.306285 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.306260    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-ee1947\"/\"openshift-service-ca.crt\""
Nov 04 03:33:22.306442 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.306427    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-ee1947\"/\"default-dockercfg-d8dk9\""
Nov 04 03:33:22.306498 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.306479    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-ee1947\"/\"premium-w1-ee1947-kserve-self-signed-certs\""
Nov 04 03:33:22.308673 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.308653    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"]
Nov 04 03:33:22.393352 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.393318    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-dshm\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.393558 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.393402    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-home\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.393558 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.393440    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3eb20c6-1760-4310-9edf-7b6392b89b63-tls-certs\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.393558 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.393457    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-tmp-dir\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.393558 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.393490    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zjf8\" (UniqueName: \"kubernetes.io/projected/b3eb20c6-1760-4310-9edf-7b6392b89b63-kube-api-access-2zjf8\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.393558 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.393519    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-model-cache\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.393877 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.393838    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-home\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.393948 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.393927    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-tmp-dir\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.393948 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.393939    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-model-cache\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.395716 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.395685    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-dshm\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.395974 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.395955    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3eb20c6-1760-4310-9edf-7b6392b89b63-tls-certs\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.455360 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.455285    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zjf8\" (UniqueName: \"kubernetes.io/projected/b3eb20c6-1760-4310-9edf-7b6392b89b63-kube-api-access-2zjf8\") pod \"premium-w1-ee1947-kserve-7cffd84d9d-2sdvt\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") " pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.509413 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.509382    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:22.889893 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:22.889869    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"]
Nov 04 03:33:22.892291 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:33:22.892264    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3eb20c6_1760_4310_9edf_7b6392b89b63.slice/crio-e1060eca4885d70555814f142623dda05cabfd9a97d3f173e5c0ccdda2f08300 WatchSource:0}: Error finding container e1060eca4885d70555814f142623dda05cabfd9a97d3f173e5c0ccdda2f08300: Status 404 returned error can't find the container with id e1060eca4885d70555814f142623dda05cabfd9a97d3f173e5c0ccdda2f08300
Nov 04 03:33:23.159946 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:23.159857    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt" event={"ID":"b3eb20c6-1760-4310-9edf-7b6392b89b63","Type":"ContainerStarted","Data":"d3a8a9a973571e44c16a5c1f8d4d409598b3d318d8118aa3d5a391198c87c71d"}
Nov 04 03:33:23.159946 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:23.159896    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt" event={"ID":"b3eb20c6-1760-4310-9edf-7b6392b89b63","Type":"ContainerStarted","Data":"e1060eca4885d70555814f142623dda05cabfd9a97d3f173e5c0ccdda2f08300"}
Nov 04 03:33:23.278508 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:23.278448    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt" podStartSLOduration=1.2784282519999999 podStartE2EDuration="1.278428252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:33:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:33:23.276565167 +0000 UTC m=+969.481165490" watchObservedRunningTime="2026-11-04 03:33:23.278428252 +0000 UTC m=+969.483028554"
Nov 04 03:33:23.509987 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:23.509910    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:28.729595 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:33:28.729558    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:33:32.510263 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:32.510228    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:32.522432 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:32.522404    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:33.223069 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:33.223028    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:33:49.156932 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.156893    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"]
Nov 04 03:33:49.160997 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.160978    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.181633 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.181604    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/672c902d-f63e-42e6-b1e2-407d6e148c04-tls-certs\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.181798 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.181760    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-home\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.181798 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.181790    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlxjb\" (UniqueName: \"kubernetes.io/projected/672c902d-f63e-42e6-b1e2-407d6e148c04-kube-api-access-nlxjb\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.181931 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.181816    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-dshm\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.181987 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.181931    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-model-cache\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.182064 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.182044    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-tmp-dir\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.268861 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.268801    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-50429e\"/\"premium-w4-50429e-kserve-self-signed-certs\""
Nov 04 03:33:49.271170 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.271148    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"]
Nov 04 03:33:49.283456 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.283422    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/672c902d-f63e-42e6-b1e2-407d6e148c04-tls-certs\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.283592 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.283535    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-home\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.283592 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.283558    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlxjb\" (UniqueName: \"kubernetes.io/projected/672c902d-f63e-42e6-b1e2-407d6e148c04-kube-api-access-nlxjb\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.283592 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.283584    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-dshm\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.283759 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.283604    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-model-cache\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.283759 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.283650    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-tmp-dir\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.284150 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.284121    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-model-cache\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.284257 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.284131    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-tmp-dir\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.284404 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.284373    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-home\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.286002 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.285981    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-dshm\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.286111 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.286041    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/672c902d-f63e-42e6-b1e2-407d6e148c04-tls-certs\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.345372 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.345336    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlxjb\" (UniqueName: \"kubernetes.io/projected/672c902d-f63e-42e6-b1e2-407d6e148c04-kube-api-access-nlxjb\") pod \"premium-w4-50429e-kserve-6f484d58fc-vk72b\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") " pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.473767 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.473681    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:49.862192 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:49.862165    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"]
Nov 04 03:33:49.864018 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:33:49.863979    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod672c902d_f63e_42e6_b1e2_407d6e148c04.slice/crio-5e7437da821733036a8737696f643c9423b14f63cf441b0e6f908e514a272a4f WatchSource:0}: Error finding container 5e7437da821733036a8737696f643c9423b14f63cf441b0e6f908e514a272a4f: Status 404 returned error can't find the container with id 5e7437da821733036a8737696f643c9423b14f63cf441b0e6f908e514a272a4f
Nov 04 03:33:50.312860 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:50.312799    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b" event={"ID":"672c902d-f63e-42e6-b1e2-407d6e148c04","Type":"ContainerStarted","Data":"414c8851e7b8a42353ba5543e8b87b93143a1a1418d981dac3578d71cad2f24a"}
Nov 04 03:33:50.313250 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:50.312876    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b" event={"ID":"672c902d-f63e-42e6-b1e2-407d6e148c04","Type":"ContainerStarted","Data":"5e7437da821733036a8737696f643c9423b14f63cf441b0e6f908e514a272a4f"}
Nov 04 03:33:50.432224 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:50.432161    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b" podStartSLOduration=1.432145529 podStartE2EDuration="1.432145529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:33:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:33:50.43060516 +0000 UTC m=+996.635205481" watchObservedRunningTime="2026-11-04 03:33:50.432145529 +0000 UTC m=+996.636745832"
Nov 04 03:33:50.473816 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:50.473773    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:58.729630 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:33:58.729589    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:33:59.474791 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:59.474753    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:33:59.487086 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:33:59.487056    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:34:00.375034 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:00.375004    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:34:12.357924 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.357889    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"]
Nov 04 03:34:12.362005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.361989    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.415152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.415123    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-ee1947-openshift-default-dockercfg-2xn5j\""
Nov 04 03:34:12.440023 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.439995    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.440176 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.440041    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.440176 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.440124    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.440249 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.440173    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpsvv\" (UniqueName: \"kubernetes.io/projected/30200904-1724-4ddd-aed5-2e9b612c26e9-kube-api-access-hpsvv\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.440287 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.440246    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.440287 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.440278    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.440379 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.440365    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.440415 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.440402    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.440450 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.440419    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30200904-1724-4ddd-aed5-2e9b612c26e9-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.468543 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.468510    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"]
Nov 04 03:34:12.541407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.541371    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.541574 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.541423    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.541574 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.541439    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30200904-1724-4ddd-aed5-2e9b612c26e9-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.541574 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.541495    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.541574 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.541532    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.541574 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.541558    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.541830 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.541603    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpsvv\" (UniqueName: \"kubernetes.io/projected/30200904-1724-4ddd-aed5-2e9b612c26e9-kube-api-access-hpsvv\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.541830 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.541626    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.541830 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.541668    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.542029 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.541871    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.542029 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.541915    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.542029 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.542016    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.542150 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.542098    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.542278 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.542254    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30200904-1724-4ddd-aed5-2e9b612c26e9-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.544013 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.543987    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.544214 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.544199    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.603179 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.603154    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.603350 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.603330    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpsvv\" (UniqueName: \"kubernetes.io/projected/30200904-1724-4ddd-aed5-2e9b612c26e9-kube-api-access-hpsvv\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:12.677382 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:12.677305    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:13.106438 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:13.106407    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"]
Nov 04 03:34:13.108751 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:34:13.108713    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30200904_1724_4ddd_aed5_2e9b612c26e9.slice/crio-4290e0cbb71b9cb85c5f5c408fd35b76c0dd39b4ca4a5b335e685ec842553f74 WatchSource:0}: Error finding container 4290e0cbb71b9cb85c5f5c408fd35b76c0dd39b4ca4a5b335e685ec842553f74: Status 404 returned error can't find the container with id 4290e0cbb71b9cb85c5f5c408fd35b76c0dd39b4ca4a5b335e685ec842553f74
Nov 04 03:34:13.111131 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:13.111097    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:34:13.111223 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:13.111181    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:34:13.111223 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:13.111208    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:34:13.448204 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:13.448087    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc" event={"ID":"30200904-1724-4ddd-aed5-2e9b612c26e9","Type":"ContainerStarted","Data":"66040061d78c078a6eef436293266a640d1d3d69d63df2ef397c445b10fba221"}
Nov 04 03:34:13.448675 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:13.448355    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc" event={"ID":"30200904-1724-4ddd-aed5-2e9b612c26e9","Type":"ContainerStarted","Data":"4290e0cbb71b9cb85c5f5c408fd35b76c0dd39b4ca4a5b335e685ec842553f74"}
Nov 04 03:34:13.572055 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:13.571997    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc" podStartSLOduration=1.571981535 podStartE2EDuration="1.571981535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:34:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:34:13.56975072 +0000 UTC m=+1019.774351042" watchObservedRunningTime="2026-11-04 03:34:13.571981535 +0000 UTC m=+1019.776581894"
Nov 04 03:34:13.677608 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:13.677573    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:14.679084 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:14.679044    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 04 03:34:14.679479 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:14.679109    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc" podUID="30200904-1724-4ddd-aed5-2e9b612c26e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 04 03:34:15.678160 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:15.678128    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 04 03:34:15.678379 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:15.678193    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc" podUID="30200904-1724-4ddd-aed5-2e9b612c26e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 04 03:34:16.678609 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:16.678574    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 04 03:34:16.679007 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:16.678637    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc" podUID="30200904-1724-4ddd-aed5-2e9b612c26e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 04 03:34:17.678502 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:17.678462    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 04 03:34:17.678681 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:17.678518    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc" podUID="30200904-1724-4ddd-aed5-2e9b612c26e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 04 03:34:18.677810 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:18.677775    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 04 03:34:18.678012 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:18.677829    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc" podUID="30200904-1724-4ddd-aed5-2e9b612c26e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 04 03:34:19.818921 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:19.818838    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:19.819456 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:19.819298    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:19.820178 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:19.820160    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:34:28.730462 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:34:28.730374    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:34:35.008857 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:35.008804    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"]
Nov 04 03:34:35.009374 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:35.009035    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5" podUID="0c64e234-343b-42ac-a4f1-44267416b6c3" containerName="istio-proxy" containerID="cri-o://9245d123ade9a6a83bd4ca9925fee32232af700552de4001414ef4ab879c0624" gracePeriod=30
Nov 04 03:34:40.273162 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.273095    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:34:40.321274 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.320154    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-credential-socket\") pod \"0c64e234-343b-42ac-a4f1-44267416b6c3\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") "
Nov 04 03:34:40.321274 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.320217    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-workload-certs\") pod \"0c64e234-343b-42ac-a4f1-44267416b6c3\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") "
Nov 04 03:34:40.321274 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.320256    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c64e234-343b-42ac-a4f1-44267416b6c3-kube-api-access-jltks\" (UniqueName: \"kubernetes.io/projected/0c64e234-343b-42ac-a4f1-44267416b6c3-kube-api-access-jltks\") pod \"0c64e234-343b-42ac-a4f1-44267416b6c3\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") "
Nov 04 03:34:40.321274 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.320344    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-envoy\") pod \"0c64e234-343b-42ac-a4f1-44267416b6c3\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") "
Nov 04 03:34:40.321274 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.320393    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-data\") pod \"0c64e234-343b-42ac-a4f1-44267416b6c3\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") "
Nov 04 03:34:40.321274 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.320447    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-workload-socket\") pod \"0c64e234-343b-42ac-a4f1-44267416b6c3\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") "
Nov 04 03:34:40.321274 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.320534    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-token\" (UniqueName: \"kubernetes.io/projected/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-token\") pod \"0c64e234-343b-42ac-a4f1-44267416b6c3\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") "
Nov 04 03:34:40.321274 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.320593    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-podinfo\") pod \"0c64e234-343b-42ac-a4f1-44267416b6c3\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") "
Nov 04 03:34:40.321274 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.320639    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c64e234-343b-42ac-a4f1-44267416b6c3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c64e234-343b-42ac-a4f1-44267416b6c3-istiod-ca-cert\") pod \"0c64e234-343b-42ac-a4f1-44267416b6c3\" (UID: \"0c64e234-343b-42ac-a4f1-44267416b6c3\") "
Nov 04 03:34:40.321831 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.321567    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-workload-socket" pod "0c64e234-343b-42ac-a4f1-44267416b6c3" (UID: "0c64e234-343b-42ac-a4f1-44267416b6c3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:34:40.321831 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.321589    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-credential-socket" pod "0c64e234-343b-42ac-a4f1-44267416b6c3" (UID: "0c64e234-343b-42ac-a4f1-44267416b6c3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:34:40.321831 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.321608    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-data" pod "0c64e234-343b-42ac-a4f1-44267416b6c3" (UID: "0c64e234-343b-42ac-a4f1-44267416b6c3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:34:40.321831 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.321647    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-workload-certs" pod "0c64e234-343b-42ac-a4f1-44267416b6c3" (UID: "0c64e234-343b-42ac-a4f1-44267416b6c3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:34:40.322098 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.322074    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c64e234-343b-42ac-a4f1-44267416b6c3-istiod-ca-cert" pod "0c64e234-343b-42ac-a4f1-44267416b6c3" (UID: "0c64e234-343b-42ac-a4f1-44267416b6c3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:34:40.323537 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.323509    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-envoy" pod "0c64e234-343b-42ac-a4f1-44267416b6c3" (UID: "0c64e234-343b-42ac-a4f1-44267416b6c3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:34:40.323931 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.323905    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c64e234-343b-42ac-a4f1-44267416b6c3-kube-api-access-jltks" pod "0c64e234-343b-42ac-a4f1-44267416b6c3" (UID: "0c64e234-343b-42ac-a4f1-44267416b6c3"). InnerVolumeSpecName "kube-api-access-jltks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:34:40.324050 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.324036    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-token" pod "0c64e234-343b-42ac-a4f1-44267416b6c3" (UID: "0c64e234-343b-42ac-a4f1-44267416b6c3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:34:40.324116 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.324052    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-podinfo" pod "0c64e234-343b-42ac-a4f1-44267416b6c3" (UID: "0c64e234-343b-42ac-a4f1-44267416b6c3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:34:40.422296 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.422254    4521 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-workload-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:40.422296 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.422289    4521 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-token\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:40.422296 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.422302    4521 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-podinfo\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:40.422562 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.422314    4521 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c64e234-343b-42ac-a4f1-44267416b6c3-istiod-ca-cert\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:40.422562 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.422326    4521 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-credential-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:40.422562 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.422340    4521 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-workload-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:40.422562 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.422352    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jltks\" (UniqueName: \"kubernetes.io/projected/0c64e234-343b-42ac-a4f1-44267416b6c3-kube-api-access-jltks\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:40.422562 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.422371    4521 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-envoy\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:40.422562 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.422384    4521 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c64e234-343b-42ac-a4f1-44267416b6c3-istio-data\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:40.601995 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.601910    4521 generic.go:356] "Generic (PLEG): container finished" podID="0c64e234-343b-42ac-a4f1-44267416b6c3" containerID="9245d123ade9a6a83bd4ca9925fee32232af700552de4001414ef4ab879c0624" exitCode=0
Nov 04 03:34:40.601995 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.601973    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"
Nov 04 03:34:40.602182 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.601999    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5" event={"ID":"0c64e234-343b-42ac-a4f1-44267416b6c3","Type":"ContainerDied","Data":"9245d123ade9a6a83bd4ca9925fee32232af700552de4001414ef4ab879c0624"}
Nov 04 03:34:40.602182 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.602083    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5" event={"ID":"0c64e234-343b-42ac-a4f1-44267416b6c3","Type":"ContainerDied","Data":"1688c937bf89398aa8b5b11f08ad8480fd8e96c60729f6c8e6881c078422e72c"}
Nov 04 03:34:40.602182 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.602107    4521 scope.go:122] "RemoveContainer" containerID="9245d123ade9a6a83bd4ca9925fee32232af700552de4001414ef4ab879c0624"
Nov 04 03:34:40.614129 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.614104    4521 scope.go:122] "RemoveContainer" containerID="9245d123ade9a6a83bd4ca9925fee32232af700552de4001414ef4ab879c0624"
Nov 04 03:34:40.614567 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:34:40.614536    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9245d123ade9a6a83bd4ca9925fee32232af700552de4001414ef4ab879c0624\": container with ID starting with 9245d123ade9a6a83bd4ca9925fee32232af700552de4001414ef4ab879c0624 not found: ID does not exist" containerID="9245d123ade9a6a83bd4ca9925fee32232af700552de4001414ef4ab879c0624"
Nov 04 03:34:40.614647 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.614580    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9245d123ade9a6a83bd4ca9925fee32232af700552de4001414ef4ab879c0624"} err="failed to get container status \"9245d123ade9a6a83bd4ca9925fee32232af700552de4001414ef4ab879c0624\": rpc error: code = NotFound desc = could not find container \"9245d123ade9a6a83bd4ca9925fee32232af700552de4001414ef4ab879c0624\": container with ID starting with 9245d123ade9a6a83bd4ca9925fee32232af700552de4001414ef4ab879c0624 not found: ID does not exist"
Nov 04 03:34:40.768091 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.768054    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"]
Nov 04 03:34:40.773250 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:40.773228    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-bvxs5"]
Nov 04 03:34:41.912387 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:34:41.912355    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c64e234-343b-42ac-a4f1-44267416b6c3" path="/var/lib/kubelet/pods/0c64e234-343b-42ac-a4f1-44267416b6c3/volumes"
Nov 04 03:34:58.730332 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:34:58.730295    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:35:05.346828 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.346793    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"]
Nov 04 03:35:05.347359 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.347342    4521 cpu_manager.go:414] "containerMap: removing container" podUID="0c64e234-343b-42ac-a4f1-44267416b6c3" containerName="istio-proxy"
Nov 04 03:35:05.347405 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.347360    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c64e234-343b-42ac-a4f1-44267416b6c3" containerName="istio-proxy"
Nov 04 03:35:05.347455 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.347443    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c64e234-343b-42ac-a4f1-44267416b6c3" containerName="istio-proxy"
Nov 04 03:35:05.350804 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.350787    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.397020 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.396978    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-tmp-dir\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.397203 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.397031    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-dshm\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.397280 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.397235    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5932542b-7aff-428d-a8e9-b8a20235035d-tls-certs\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.397322 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.397299    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-model-cache\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.397374 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.397336    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndbkf\" (UniqueName: \"kubernetes.io/projected/5932542b-7aff-428d-a8e9-b8a20235035d-kube-api-access-ndbkf\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.397450 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.397414    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-home\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.422342 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.422314    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-fba151\"/\"openshift-service-ca.crt\""
Nov 04 03:35:05.454000 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.453966    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-fba151\"/\"kube-root-ca.crt\""
Nov 04 03:35:05.454155 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.454112    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-fba151\"/\"distinct-w2-fba151-kserve-self-signed-certs\""
Nov 04 03:35:05.454231 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.454203    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-fba151\"/\"default-dockercfg-s4t97\""
Nov 04 03:35:05.456641 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.456618    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"]
Nov 04 03:35:05.498414 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.498381    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-tmp-dir\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.498592 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.498428    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-dshm\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.498592 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.498531    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5932542b-7aff-428d-a8e9-b8a20235035d-tls-certs\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.498592 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.498549    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-model-cache\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.498768 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.498591    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndbkf\" (UniqueName: \"kubernetes.io/projected/5932542b-7aff-428d-a8e9-b8a20235035d-kube-api-access-ndbkf\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.498768 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.498632    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-home\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.498900 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.498865    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-tmp-dir\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.498961 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.498930    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-model-cache\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.499042 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.499019    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-home\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.501093 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.501064    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5932542b-7aff-428d-a8e9-b8a20235035d-tls-certs\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.501216 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.501146    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-dshm\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.560754 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.560718    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndbkf\" (UniqueName: \"kubernetes.io/projected/5932542b-7aff-428d-a8e9-b8a20235035d-kube-api-access-ndbkf\") pod \"distinct-w2-fba151-kserve-6b976ff9f5-pdcf7\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:05.663400 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:05.663322    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:06.048415 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:06.048389    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"]
Nov 04 03:35:06.050141 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:35:06.050115    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5932542b_7aff_428d_a8e9_b8a20235035d.slice/crio-5d90ee505f747eeebb38b9296e58369f8b1b17e8ae4a99ce70a4748fd038574d WatchSource:0}: Error finding container 5d90ee505f747eeebb38b9296e58369f8b1b17e8ae4a99ce70a4748fd038574d: Status 404 returned error can't find the container with id 5d90ee505f747eeebb38b9296e58369f8b1b17e8ae4a99ce70a4748fd038574d
Nov 04 03:35:06.749231 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:06.749187    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7" event={"ID":"5932542b-7aff-428d-a8e9-b8a20235035d","Type":"ContainerStarted","Data":"02a22995f7a4748e909b94da8e681573122786d0c00fa859235609ceebe8039e"}
Nov 04 03:35:06.749648 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:06.749241    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7" event={"ID":"5932542b-7aff-428d-a8e9-b8a20235035d","Type":"ContainerStarted","Data":"5d90ee505f747eeebb38b9296e58369f8b1b17e8ae4a99ce70a4748fd038574d"}
Nov 04 03:35:06.868302 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:06.868248    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7" podStartSLOduration=1.8682344199999998 podStartE2EDuration="1.86823442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:35:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:35:06.866868943 +0000 UTC m=+1073.071469240" watchObservedRunningTime="2026-11-04 03:35:06.86823442 +0000 UTC m=+1073.072834722"
Nov 04 03:35:07.663646 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:07.663613    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:15.663653 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:15.663610    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:15.676442 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:15.676413    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:15.814213 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:15.814179    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:35:28.729252 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:35:28.729217    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:35:57.883876 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:57.883771    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"]
Nov 04 03:35:57.888041 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:57.888024    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:57.981825 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:57.981785    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29ed832f-92f7-4a13-9a94-b65f05ab260c-tls-certs\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:57.982008 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:57.981930    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-dshm\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:57.982081 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:57.982064    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-model-cache\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:57.982134 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:57.982120    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-tmp-dir\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:57.982245 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:57.982222    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqfrn\" (UniqueName: \"kubernetes.io/projected/29ed832f-92f7-4a13-9a94-b65f05ab260c-kube-api-access-tqfrn\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:57.982313 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:57.982285    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-home\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:57.994084 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:57.994058    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-ee1947\"/\"distinct-w1-ee1947-kserve-self-signed-certs\""
Nov 04 03:35:57.996464 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:57.996442    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"]
Nov 04 03:35:58.083134 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:58.083101    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-model-cache\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:58.083328 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:58.083150    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-tmp-dir\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:58.083328 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:58.083196    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqfrn\" (UniqueName: \"kubernetes.io/projected/29ed832f-92f7-4a13-9a94-b65f05ab260c-kube-api-access-tqfrn\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:58.083328 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:58.083240    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-home\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:58.083328 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:58.083278    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29ed832f-92f7-4a13-9a94-b65f05ab260c-tls-certs\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:58.083570 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:58.083338    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-dshm\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:58.083635 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:58.083588    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-model-cache\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:58.083692 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:58.083645    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-tmp-dir\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:58.083692 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:58.083684    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-home\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:58.085652 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:58.085629    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-dshm\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:58.085745 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:58.085721    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29ed832f-92f7-4a13-9a94-b65f05ab260c-tls-certs\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:58.144311 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:58.144251    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqfrn\" (UniqueName: \"kubernetes.io/projected/29ed832f-92f7-4a13-9a94-b65f05ab260c-kube-api-access-tqfrn\") pod \"distinct-w1-ee1947-kserve-5dc45d786f-kcpqj\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:58.200443 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:58.200396    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:35:58.582685 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:58.582655    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"]
Nov 04 03:35:58.584445 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:35:58.584420    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29ed832f_92f7_4a13_9a94_b65f05ab260c.slice/crio-b92ec904227962763701b2cf7153eab67debced72fd6c06a8c8f1570aa529ae2 WatchSource:0}: Error finding container b92ec904227962763701b2cf7153eab67debced72fd6c06a8c8f1570aa529ae2: Status 404 returned error can't find the container with id b92ec904227962763701b2cf7153eab67debced72fd6c06a8c8f1570aa529ae2
Nov 04 03:35:58.730284 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:35:58.730242    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:35:59.040551 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:59.040517    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj" event={"ID":"29ed832f-92f7-4a13-9a94-b65f05ab260c","Type":"ContainerStarted","Data":"bde1f3573a464a2d2c8bf4a5e0c0f7eafce9b00756b0641e5d1d0bb56d57856c"}
Nov 04 03:35:59.040987 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:59.040560    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj" event={"ID":"29ed832f-92f7-4a13-9a94-b65f05ab260c","Type":"ContainerStarted","Data":"b92ec904227962763701b2cf7153eab67debced72fd6c06a8c8f1570aa529ae2"}
Nov 04 03:35:59.195859 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:59.195787    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj" podStartSLOduration=2.1957707109999998 podStartE2EDuration="2.195770711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:35:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:35:59.194310315 +0000 UTC m=+1125.398910840" watchObservedRunningTime="2026-11-04 03:35:59.195770711 +0000 UTC m=+1125.400371012"
Nov 04 03:35:59.200518 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:35:59.200495    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:36:08.201150 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:08.201112    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:36:08.213708 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:08.213681    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:36:09.113809 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:09.113781    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:36:28.494945 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.494901    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"]
Nov 04 03:36:28.500207 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.500182    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.607644 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.607605    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"]
Nov 04 03:36:28.624639 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.624606    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c25gn\" (UniqueName: \"kubernetes.io/projected/efc6e068-c4fc-4559-a531-d62af03cbc9b-kube-api-access-c25gn\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.624894 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.624869    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.624998 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.624901    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.625068 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.625044    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.625152 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.625136    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.625191 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.625169    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efc6e068-c4fc-4559-a531-d62af03cbc9b-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.625252 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.625240    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.625316 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.625305    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.625352 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.625340    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.726484 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.726450    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.726662 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.726492    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efc6e068-c4fc-4559-a531-d62af03cbc9b-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.726662 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.726550    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.726759 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.726696    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.726811 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.726757    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.726878 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.726828    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c25gn\" (UniqueName: \"kubernetes.io/projected/efc6e068-c4fc-4559-a531-d62af03cbc9b-kube-api-access-c25gn\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.726943 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.726885    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.726943 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.726920    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.727060 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.727044    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.727130 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.727063    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.727130 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.726919    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.727292 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.727262    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.727292 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.727279    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efc6e068-c4fc-4559-a531-d62af03cbc9b-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.727489 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.727464    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.729447 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.729421    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.729985 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.729961    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.731557 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:36:28.731522    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:36:28.788262 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.788233    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.788389 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.788241    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c25gn\" (UniqueName: \"kubernetes.io/projected/efc6e068-c4fc-4559-a531-d62af03cbc9b-kube-api-access-c25gn\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:28.814710 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:28.814686    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:29.248379 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:29.248352    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"]
Nov 04 03:36:29.250485 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:36:29.250455    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefc6e068_c4fc_4559_a531_d62af03cbc9b.slice/crio-1641eb3d73d0c5c7d91069cf20e5135911b9fc11a74414aab3232b2dcb0dbf7f WatchSource:0}: Error finding container 1641eb3d73d0c5c7d91069cf20e5135911b9fc11a74414aab3232b2dcb0dbf7f: Status 404 returned error can't find the container with id 1641eb3d73d0c5c7d91069cf20e5135911b9fc11a74414aab3232b2dcb0dbf7f
Nov 04 03:36:29.252716 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:29.252681    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:36:29.252808 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:29.252749    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:36:29.252808 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:29.252777    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:36:30.230066 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:30.230028    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm" event={"ID":"efc6e068-c4fc-4559-a531-d62af03cbc9b","Type":"ContainerStarted","Data":"a9c7da8a88d1fba93b2262df8ce38e93dc8c50b66a100d614734f8027249ed11"}
Nov 04 03:36:30.230480 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:30.230086    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm" event={"ID":"efc6e068-c4fc-4559-a531-d62af03cbc9b","Type":"ContainerStarted","Data":"1641eb3d73d0c5c7d91069cf20e5135911b9fc11a74414aab3232b2dcb0dbf7f"}
Nov 04 03:36:30.815176 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:30.815139    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:30.817144 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:30.817109    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 03:36:30.817293 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:30.817157    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm" podUID="efc6e068-c4fc-4559-a531-d62af03cbc9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 03:36:31.815928 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:31.815885    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 03:36:31.816346 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:31.815950    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm" podUID="efc6e068-c4fc-4559-a531-d62af03cbc9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 03:36:32.815477 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:32.815436    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 03:36:32.815663 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:32.815504    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm" podUID="efc6e068-c4fc-4559-a531-d62af03cbc9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 03:36:33.816056 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:33.816016    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 03:36:33.816573 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:33.816083    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm" podUID="efc6e068-c4fc-4559-a531-d62af03cbc9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 03:36:34.816042 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:34.816007    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 03:36:34.816219 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:34.816082    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm" podUID="efc6e068-c4fc-4559-a531-d62af03cbc9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 03:36:35.815981 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:35.815946    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 03:36:35.816163 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:35.816000    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm" podUID="efc6e068-c4fc-4559-a531-d62af03cbc9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 03:36:36.819425 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:36.819397    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:36.819907 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:36.819679    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:36.820356 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:36.820337    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:36:37.044589 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:36:37.044533    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm" podStartSLOduration=9.044518771 podStartE2EDuration="9.044518771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:36:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:36:30.405176393 +0000 UTC m=+1156.609776723" watchObservedRunningTime="2026-11-04 03:36:37.044518771 +0000 UTC m=+1163.249119073"
Nov 04 03:36:58.728547 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:36:58.728510    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:37:16.179407 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.179372    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"]
Nov 04 03:37:16.183556 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.183538    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.240051 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.239522    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-ee1947\"/\"distinct-2-w1-ee1947-kserve-self-signed-certs\""
Nov 04 03:37:16.242787 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.242756    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"]
Nov 04 03:37:16.303055 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.303021    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-tmp-dir\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.303218 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.303144    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-model-cache\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.303272 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.303225    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-tls-certs\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.303316 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.303300    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-dshm\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.303470 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.303448    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4sbj\" (UniqueName: \"kubernetes.io/projected/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-kube-api-access-r4sbj\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.303560 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.303545    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-home\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.404734 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.404695    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4sbj\" (UniqueName: \"kubernetes.io/projected/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-kube-api-access-r4sbj\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.404921 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.404778    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-home\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.404921 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.404830    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-tmp-dir\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.404921 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.404886    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-model-cache\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.405088 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.404945    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-tls-certs\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.405088 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.404979    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-dshm\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.405324 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.405299    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-home\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.405402 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.405328    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-tmp-dir\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.405402 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.405364    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-model-cache\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.407320 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.407297    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-dshm\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.407524 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.407507    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-tls-certs\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.467171 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.467099    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4sbj\" (UniqueName: \"kubernetes.io/projected/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-kube-api-access-r4sbj\") pod \"distinct-2-w1-ee1947-kserve-665fb85566-ds8pc\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") " pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.496237 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.496205    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:16.872509 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:16.872484    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"]
Nov 04 03:37:16.873744 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:37:16.873719    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8ab2132_5f2b_43ef_8a57_acfa16dc8090.slice/crio-2630be14dba77a4be05e37ab18fa7c4807d2bf360158ec58f575eba644724499 WatchSource:0}: Error finding container 2630be14dba77a4be05e37ab18fa7c4807d2bf360158ec58f575eba644724499: Status 404 returned error can't find the container with id 2630be14dba77a4be05e37ab18fa7c4807d2bf360158ec58f575eba644724499
Nov 04 03:37:17.485995 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:17.485958    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc" event={"ID":"b8ab2132-5f2b-43ef-8a57-acfa16dc8090","Type":"ContainerStarted","Data":"e9968b85da209b9578bc2f2b02b9ff092dcfeaf34e462a2ed840a3c899777e91"}
Nov 04 03:37:17.486400 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:17.486004    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc" event={"ID":"b8ab2132-5f2b-43ef-8a57-acfa16dc8090","Type":"ContainerStarted","Data":"2630be14dba77a4be05e37ab18fa7c4807d2bf360158ec58f575eba644724499"}
Nov 04 03:37:17.496427 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:17.496405    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:17.607228 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:17.607171    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc" podStartSLOduration=1.607156021 podStartE2EDuration="1.607156021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:37:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:37:17.60487063 +0000 UTC m=+1203.809470929" watchObservedRunningTime="2026-11-04 03:37:17.607156021 +0000 UTC m=+1203.811756324"
Nov 04 03:37:26.496613 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:26.496520    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:26.508908 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:26.508883    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:26.546048 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:26.546011    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:37:28.731645 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:37:28.731600    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:37:33.188321 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:33.188285    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"]
Nov 04 03:37:33.191985 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:33.191968    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"
Nov 04 03:37:33.246818 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:33.246790    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-1bf4bf\""
Nov 04 03:37:33.297234 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:33.297202    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-1bf4bf-dockercfg-m72rs\""
Nov 04 03:37:33.297646 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:33.297621    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz62d\" (UniqueName: \"kubernetes.io/projected/3483c5f0-468c-4b53-8efb-77788eae77a8-kube-api-access-bz62d\") pod \"payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn\" (UID: \"3483c5f0-468c-4b53-8efb-77788eae77a8\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"
Nov 04 03:37:33.297700 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:33.297660    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3483c5f0-468c-4b53-8efb-77788eae77a8-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn\" (UID: \"3483c5f0-468c-4b53-8efb-77788eae77a8\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"
Nov 04 03:37:33.299759 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:33.299734    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"]
Nov 04 03:37:33.398873 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:33.398817    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz62d\" (UniqueName: \"kubernetes.io/projected/3483c5f0-468c-4b53-8efb-77788eae77a8-kube-api-access-bz62d\") pod \"payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn\" (UID: \"3483c5f0-468c-4b53-8efb-77788eae77a8\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"
Nov 04 03:37:33.398873 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:33.398870    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3483c5f0-468c-4b53-8efb-77788eae77a8-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn\" (UID: \"3483c5f0-468c-4b53-8efb-77788eae77a8\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"
Nov 04 03:37:33.399616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:33.399596    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3483c5f0-468c-4b53-8efb-77788eae77a8-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn\" (UID: \"3483c5f0-468c-4b53-8efb-77788eae77a8\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"
Nov 04 03:37:33.465167 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:33.465080    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz62d\" (UniqueName: \"kubernetes.io/projected/3483c5f0-468c-4b53-8efb-77788eae77a8-kube-api-access-bz62d\") pod \"payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn\" (UID: \"3483c5f0-468c-4b53-8efb-77788eae77a8\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"
Nov 04 03:37:33.506307 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:33.506273    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"
Nov 04 03:37:33.888655 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:33.888621    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"]
Nov 04 03:37:33.889199 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:37:33.889169    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3483c5f0_468c_4b53_8efb_77788eae77a8.slice/crio-5fa9206222cf3f8d7e81fc4d16aeace3be0fa146c7a43ccb91f38c01377ff29a WatchSource:0}: Error finding container 5fa9206222cf3f8d7e81fc4d16aeace3be0fa146c7a43ccb91f38c01377ff29a: Status 404 returned error can't find the container with id 5fa9206222cf3f8d7e81fc4d16aeace3be0fa146c7a43ccb91f38c01377ff29a
Nov 04 03:37:34.584751 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:34.584717    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn" event={"ID":"3483c5f0-468c-4b53-8efb-77788eae77a8","Type":"ContainerStarted","Data":"16d230f15333cff75956e8eca7d6d2eacc5a34465f399dca148ddfa043741a5d"}
Nov 04 03:37:34.585294 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:34.584769    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn" event={"ID":"3483c5f0-468c-4b53-8efb-77788eae77a8","Type":"ContainerStarted","Data":"5fa9206222cf3f8d7e81fc4d16aeace3be0fa146c7a43ccb91f38c01377ff29a"}
Nov 04 03:37:34.585294 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:34.584799    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"
Nov 04 03:37:34.707868 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:34.707687    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn" podStartSLOduration=1.707666632 podStartE2EDuration="1.707666632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:37:34.702307319 +0000 UTC m=+1220.906907653" watchObservedRunningTime="2026-11-04 03:37:34.707666632 +0000 UTC m=+1220.912266934"
Nov 04 03:37:45.593572 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:45.593543    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"
Nov 04 03:37:52.577353 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:52.577318    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"]
Nov 04 03:37:52.581762 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:52.581738    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"
Nov 04 03:37:52.636412 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:52.636377    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-59c311\""
Nov 04 03:37:52.644260 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:52.644225    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95615e76-0669-492a-b8a5-9b48b7fb1b15-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv\" (UID: \"95615e76-0669-492a-b8a5-9b48b7fb1b15\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"
Nov 04 03:37:52.644532 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:52.644508    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ztpz\" (UniqueName: \"kubernetes.io/projected/95615e76-0669-492a-b8a5-9b48b7fb1b15-kube-api-access-5ztpz\") pod \"payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv\" (UID: \"95615e76-0669-492a-b8a5-9b48b7fb1b15\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"
Nov 04 03:37:52.686680 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:52.686648    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-59c311-dockercfg-h6rml\""
Nov 04 03:37:52.689248 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:52.689226    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"]
Nov 04 03:37:52.745952 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:52.745914    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ztpz\" (UniqueName: \"kubernetes.io/projected/95615e76-0669-492a-b8a5-9b48b7fb1b15-kube-api-access-5ztpz\") pod \"payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv\" (UID: \"95615e76-0669-492a-b8a5-9b48b7fb1b15\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"
Nov 04 03:37:52.746117 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:52.746043    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95615e76-0669-492a-b8a5-9b48b7fb1b15-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv\" (UID: \"95615e76-0669-492a-b8a5-9b48b7fb1b15\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"
Nov 04 03:37:52.746724 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:52.746705    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95615e76-0669-492a-b8a5-9b48b7fb1b15-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv\" (UID: \"95615e76-0669-492a-b8a5-9b48b7fb1b15\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"
Nov 04 03:37:52.807796 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:52.807766    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ztpz\" (UniqueName: \"kubernetes.io/projected/95615e76-0669-492a-b8a5-9b48b7fb1b15-kube-api-access-5ztpz\") pod \"payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv\" (UID: \"95615e76-0669-492a-b8a5-9b48b7fb1b15\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"
Nov 04 03:37:52.897313 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:52.897229    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"
Nov 04 03:37:53.272732 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:53.272704    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"]
Nov 04 03:37:53.274395 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:37:53.274362    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95615e76_0669_492a_b8a5_9b48b7fb1b15.slice/crio-97ca198a055d74ed696ba022d70db553b301d9f4c985b872076d5b9cbbb8fdd0 WatchSource:0}: Error finding container 97ca198a055d74ed696ba022d70db553b301d9f4c985b872076d5b9cbbb8fdd0: Status 404 returned error can't find the container with id 97ca198a055d74ed696ba022d70db553b301d9f4c985b872076d5b9cbbb8fdd0
Nov 04 03:37:53.694780 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:53.694689    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv" event={"ID":"95615e76-0669-492a-b8a5-9b48b7fb1b15","Type":"ContainerStarted","Data":"764d07a8060e82091d9dd08cd58fe5294e1fce4277ae76547941f2148792412e"}
Nov 04 03:37:53.694780 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:53.694775    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv" event={"ID":"95615e76-0669-492a-b8a5-9b48b7fb1b15","Type":"ContainerStarted","Data":"97ca198a055d74ed696ba022d70db553b301d9f4c985b872076d5b9cbbb8fdd0"}
Nov 04 03:37:53.695270 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:53.694811    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"
Nov 04 03:37:53.812683 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:53.812625    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv" podStartSLOduration=1.8126117640000001 podStartE2EDuration="1.812611764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:37:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:37:53.811113009 +0000 UTC m=+1240.015713330" watchObservedRunningTime="2026-11-04 03:37:53.812611764 +0000 UTC m=+1240.017212067"
Nov 04 03:37:58.730599 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:37:58.730557    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:37:59.219130 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.219095    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"]
Nov 04 03:37:59.223224 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.223200    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.327657 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.327622    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjcbp\" (UniqueName: \"kubernetes.io/projected/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-kube-api-access-zjcbp\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.327837 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.327735    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.327837 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.327785    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.327837 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.327804    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.327957 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.327868    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.327957 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.327904    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.328027 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.327968    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.328027 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.328002    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.328090 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.328049    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.429079 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.429041    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.429278 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.429092    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.429278 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.429198    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.429411 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.429329    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjcbp\" (UniqueName: \"kubernetes.io/projected/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-kube-api-access-zjcbp\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.429469 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.429423    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.429529 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.429482    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.429584 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.429544    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.429639 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.429580    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.429639 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.429612    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.429639 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.429544    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.429639 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.429633    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.429967 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.429943    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.430189 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.430169    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.430592 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.430572    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.432085 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.432054    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.432176 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.432117    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.439595 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.439570    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"]
Nov 04 03:37:59.491012 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.490936    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.491012 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.491002    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjcbp\" (UniqueName: \"kubernetes.io/projected/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-kube-api-access-zjcbp\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.537302 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.537265    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:37:59.971831 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.971801    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"]
Nov 04 03:37:59.973559 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:37:59.973531    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod934b3519_cc81_4ea9_b8bd_9e962e9fcacb.slice/crio-f87c82347c1e7fb4eb9d95f7c26d9701d50347d60323cddedff5f5f372d7a8f3 WatchSource:0}: Error finding container f87c82347c1e7fb4eb9d95f7c26d9701d50347d60323cddedff5f5f372d7a8f3: Status 404 returned error can't find the container with id f87c82347c1e7fb4eb9d95f7c26d9701d50347d60323cddedff5f5f372d7a8f3
Nov 04 03:37:59.976033 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.975999    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:59.976131 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.976074    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:59.976131 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:37:59.976104    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:38:00.738888 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:00.738854    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd" event={"ID":"934b3519-cc81-4ea9-b8bd-9e962e9fcacb","Type":"ContainerStarted","Data":"6db1f1734c6cfcd5d2162b8596c356d76f555a121afebafe31a59452cd80b1a7"}
Nov 04 03:38:00.739074 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:00.738904    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd" event={"ID":"934b3519-cc81-4ea9-b8bd-9e962e9fcacb","Type":"ContainerStarted","Data":"f87c82347c1e7fb4eb9d95f7c26d9701d50347d60323cddedff5f5f372d7a8f3"}
Nov 04 03:38:00.968501 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:00.968446    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd" podStartSLOduration=1.9684313169999998 podStartE2EDuration="1.968431317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:37:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:38:00.916414666 +0000 UTC m=+1247.121014987" watchObservedRunningTime="2026-11-04 03:38:00.968431317 +0000 UTC m=+1247.173031618"
Nov 04 03:38:01.537452 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:01.537414    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:38:01.539321 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:01.539291    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 03:38:01.539441 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:01.539359    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd" podUID="934b3519-cc81-4ea9-b8bd-9e962e9fcacb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 03:38:02.538310 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:02.538267    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 03:38:02.538822 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:02.538338    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd" podUID="934b3519-cc81-4ea9-b8bd-9e962e9fcacb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 03:38:03.538084 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:03.538043    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 03:38:03.538280 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:03.538118    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd" podUID="934b3519-cc81-4ea9-b8bd-9e962e9fcacb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 03:38:04.538093 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:04.538053    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 03:38:04.538572 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:04.538113    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd" podUID="934b3519-cc81-4ea9-b8bd-9e962e9fcacb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 03:38:04.712706 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:04.712667    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"
Nov 04 03:38:05.538354 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:05.538317    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 03:38:05.538780 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:05.538370    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd" podUID="934b3519-cc81-4ea9-b8bd-9e962e9fcacb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 03:38:06.537805 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:06.537757    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 03:38:06.538005 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:06.537827    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd" podUID="934b3519-cc81-4ea9-b8bd-9e962e9fcacb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 03:38:07.542884 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:07.542852    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:38:07.543378 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:07.543356    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:38:07.544223 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:07.544200    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:38:17.551663 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:17.551627    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"]
Nov 04 03:38:17.555979 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:17.555960    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:38:17.608715 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:17.608686    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-6d170d-serving-cert\""
Nov 04 03:38:17.661984 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:17.661953    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"]
Nov 04 03:38:17.664127 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:17.664096    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-maas-api-tls\") pod \"maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl\" (UID: \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:38:17.664243 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:17.664172    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-metrics-certs\") pod \"maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl\" (UID: \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:38:17.664337 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:17.664321    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pd6j\" (UniqueName: \"kubernetes.io/projected/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-kube-api-access-2pd6j\") pod \"maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl\" (UID: \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:38:17.664418 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:17.664405    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl\" (UID: \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:38:17.765895 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:17.765864    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-maas-api-tls\") pod \"maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl\" (UID: \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:38:17.766092 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:17.765901    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-metrics-certs\") pod \"maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl\" (UID: \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:38:17.766092 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:17.765967    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pd6j\" (UniqueName: \"kubernetes.io/projected/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-kube-api-access-2pd6j\") pod \"maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl\" (UID: \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:38:17.766092 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:17.766001    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl\" (UID: \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:38:17.766092 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:38:17.766004    4521 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-serving-cert: secret "maas-api-e2e-extmod-a-6d170d-serving-cert" not found
Nov 04 03:38:17.766092 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:38:17.766076    4521 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-maas-api-tls podName:a9ea7e36-1384-4ab8-9f5c-a5562faba8c2 nodeName:}" failed. No retries permitted until 2026-11-04 03:38:18.266053203 +0000 UTC m=+1264.470653497 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-maas-api-tls") pod "maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl" (UID: "a9ea7e36-1384-4ab8-9f5c-a5562faba8c2") : secret "maas-api-e2e-extmod-a-6d170d-serving-cert" not found
Nov 04 03:38:17.766719 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:17.766695    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl\" (UID: \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:38:17.768357 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:17.768338    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-metrics-certs\") pod \"maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl\" (UID: \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:38:17.827813 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:17.827752    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pd6j\" (UniqueName: \"kubernetes.io/projected/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-kube-api-access-2pd6j\") pod \"maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl\" (UID: \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:38:18.271911 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:18.271877    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-maas-api-tls\") pod \"maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl\" (UID: \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:38:18.274269 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:18.274249    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-maas-api-tls\") pod \"maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl\" (UID: \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:38:18.470063 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:18.470025    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:38:18.852107 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:18.852081    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"]
Nov 04 03:38:18.854151 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:38:18.854121    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9ea7e36_1384_4ab8_9f5c_a5562faba8c2.slice/crio-b0837bd5aba1f0877af0c0560bb9c0be3dbffa71ed95e2003f0c7d442689333d WatchSource:0}: Error finding container b0837bd5aba1f0877af0c0560bb9c0be3dbffa71ed95e2003f0c7d442689333d: Status 404 returned error can't find the container with id b0837bd5aba1f0877af0c0560bb9c0be3dbffa71ed95e2003f0c7d442689333d
Nov 04 03:38:18.855586 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:18.855564    4521 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:38:19.851566 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:19.851535    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl" event={"ID":"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2","Type":"ContainerStarted","Data":"b0837bd5aba1f0877af0c0560bb9c0be3dbffa71ed95e2003f0c7d442689333d"}
Nov 04 03:38:20.859072 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:20.859037    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl" event={"ID":"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2","Type":"ContainerStarted","Data":"eb36dcb39a77e726e639ad1a2ab12a0eb7c28c93bbd796efc65efafcdc4cf063"}
Nov 04 03:38:20.859453 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:20.859165    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:38:20.980456 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:20.980395    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl" podStartSLOduration=3.021634882 podStartE2EDuration="3.980382229s" totalImagesPullingTime="958.747347ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:38:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:38:20.977707194 +0000 UTC m=+1267.182307531" watchObservedRunningTime="2026-11-04 03:38:20.980382229 +0000 UTC m=+1267.184982531"
Nov 04 03:38:26.869476 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:26.869443    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:38:28.730093 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:38:28.730052    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:38:32.313382 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.313339    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"]
Nov 04 03:38:32.318978 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.318950    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.372350 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.372316    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-092f8c-openshift-default-dockercfg-9bd4m\""
Nov 04 03:38:32.427295 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.427258    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"]
Nov 04 03:38:32.445760 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.445726    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8901771-3083-4017-93f8-331b57fe7b94-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.446019 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.445990    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.446097 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.446069    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8901771-3083-4017-93f8-331b57fe7b94-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.446268 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.446239    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.446348 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.446298    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.446348 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.446331    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8901771-3083-4017-93f8-331b57fe7b94-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.446462 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.446411    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.446462 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.446453    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.446565 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.446484    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr4h9\" (UniqueName: \"kubernetes.io/projected/a8901771-3083-4017-93f8-331b57fe7b94-kube-api-access-pr4h9\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.547219 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.547185    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.547417 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.547253    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.547417 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.547285    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8901771-3083-4017-93f8-331b57fe7b94-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.547417 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.547344    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.547591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.547495    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.547591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.547561    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr4h9\" (UniqueName: \"kubernetes.io/projected/a8901771-3083-4017-93f8-331b57fe7b94-kube-api-access-pr4h9\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.547697 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.547650    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8901771-3083-4017-93f8-331b57fe7b94-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.547697 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.547665    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.547697 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.547688    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.547869 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.547762    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8901771-3083-4017-93f8-331b57fe7b94-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.548420 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.548368    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.548673 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.548645    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8901771-3083-4017-93f8-331b57fe7b94-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.548965 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.548938    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.549061 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.548977    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.550856 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.550787    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8901771-3083-4017-93f8-331b57fe7b94-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.551338 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.551314    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.609999 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.609926    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8901771-3083-4017-93f8-331b57fe7b94-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.610231 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.610204    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr4h9\" (UniqueName: \"kubernetes.io/projected/a8901771-3083-4017-93f8-331b57fe7b94-kube-api-access-pr4h9\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:32.636982 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:32.636946    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:33.073855 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:33.073815    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"]
Nov 04 03:38:33.075470 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:38:33.075444    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8901771_3083_4017_93f8_331b57fe7b94.slice/crio-928aded69819986bb75970f7d2ca9ed0a3306301a52a3fb849eaf80e7ace6f6e WatchSource:0}: Error finding container 928aded69819986bb75970f7d2ca9ed0a3306301a52a3fb849eaf80e7ace6f6e: Status 404 returned error can't find the container with id 928aded69819986bb75970f7d2ca9ed0a3306301a52a3fb849eaf80e7ace6f6e
Nov 04 03:38:33.077976 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:33.077940    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:38:33.078045 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:33.078014    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:38:33.078083 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:33.078044    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:38:33.937511 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:33.937476    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr" event={"ID":"a8901771-3083-4017-93f8-331b57fe7b94","Type":"ContainerStarted","Data":"abac4d131f1f1efab5864ffdcf6e7ed665c1fd898330b4ceb74eed347b23b949"}
Nov 04 03:38:33.937911 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:33.937528    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr" event={"ID":"a8901771-3083-4017-93f8-331b57fe7b94","Type":"ContainerStarted","Data":"928aded69819986bb75970f7d2ca9ed0a3306301a52a3fb849eaf80e7ace6f6e"}
Nov 04 03:38:34.071087 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:34.071028    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr" podStartSLOduration=2.071012536 podStartE2EDuration="2.071012536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:38:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:38:34.068756614 +0000 UTC m=+1280.273356916" watchObservedRunningTime="2026-11-04 03:38:34.071012536 +0000 UTC m=+1280.275612844"
Nov 04 03:38:34.637915 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:34.637879    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:34.639366 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:34.639338    4521 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 03:38:34.639500 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:34.639391    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr" podUID="a8901771-3083-4017-93f8-331b57fe7b94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 03:38:35.638131 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:35.638096    4521 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 03:38:35.638640 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:35.638152    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr" podUID="a8901771-3083-4017-93f8-331b57fe7b94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 03:38:36.638186 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:36.638150    4521 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 03:38:36.638605 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:36.638216    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr" podUID="a8901771-3083-4017-93f8-331b57fe7b94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 03:38:37.637974 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:37.637937    4521 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 03:38:37.638161 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:37.638003    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr" podUID="a8901771-3083-4017-93f8-331b57fe7b94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 03:38:37.856967 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:37.856934    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"]
Nov 04 03:38:37.861310 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:37.861290    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:38:37.916965 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:37.916881    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-59c311-serving-cert\""
Nov 04 03:38:37.921101 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:37.921074    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"]
Nov 04 03:38:38.037033 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:38.036998    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf30d8d-9a93-4298-9ee3-92887e638513-maas-api-tls\") pod \"maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj\" (UID: \"abf30d8d-9a93-4298-9ee3-92887e638513\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:38:38.037264 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:38.037229    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjq8r\" (UniqueName: \"kubernetes.io/projected/abf30d8d-9a93-4298-9ee3-92887e638513-kube-api-access-zjq8r\") pod \"maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj\" (UID: \"abf30d8d-9a93-4298-9ee3-92887e638513\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:38:38.037444 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:38.037421    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf30d8d-9a93-4298-9ee3-92887e638513-metrics-certs\") pod \"maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj\" (UID: \"abf30d8d-9a93-4298-9ee3-92887e638513\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:38:38.037524 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:38.037475    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf30d8d-9a93-4298-9ee3-92887e638513-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj\" (UID: \"abf30d8d-9a93-4298-9ee3-92887e638513\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:38:38.140233 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:38.140196    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf30d8d-9a93-4298-9ee3-92887e638513-metrics-certs\") pod \"maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj\" (UID: \"abf30d8d-9a93-4298-9ee3-92887e638513\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:38:38.140424 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:38.140262    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf30d8d-9a93-4298-9ee3-92887e638513-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj\" (UID: \"abf30d8d-9a93-4298-9ee3-92887e638513\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:38:38.140424 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:38.140396    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf30d8d-9a93-4298-9ee3-92887e638513-maas-api-tls\") pod \"maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj\" (UID: \"abf30d8d-9a93-4298-9ee3-92887e638513\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:38:38.140537 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:38.140420    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjq8r\" (UniqueName: \"kubernetes.io/projected/abf30d8d-9a93-4298-9ee3-92887e638513-kube-api-access-zjq8r\") pod \"maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj\" (UID: \"abf30d8d-9a93-4298-9ee3-92887e638513\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:38:38.140591 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:38:38.140553    4521 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-serving-cert: secret "maas-api-e2e-extmod-b-59c311-serving-cert" not found
Nov 04 03:38:38.140656 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:38:38.140641    4521 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abf30d8d-9a93-4298-9ee3-92887e638513-maas-api-tls podName:abf30d8d-9a93-4298-9ee3-92887e638513 nodeName:}" failed. No retries permitted until 2026-11-04 03:38:38.640618271 +0000 UTC m=+1284.845218567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/abf30d8d-9a93-4298-9ee3-92887e638513-maas-api-tls") pod "maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj" (UID: "abf30d8d-9a93-4298-9ee3-92887e638513") : secret "maas-api-e2e-extmod-b-59c311-serving-cert" not found
Nov 04 03:38:38.141100 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:38.141074    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf30d8d-9a93-4298-9ee3-92887e638513-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj\" (UID: \"abf30d8d-9a93-4298-9ee3-92887e638513\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:38:38.143003 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:38.142975    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf30d8d-9a93-4298-9ee3-92887e638513-metrics-certs\") pod \"maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj\" (UID: \"abf30d8d-9a93-4298-9ee3-92887e638513\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:38:38.211649 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:38.211569    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjq8r\" (UniqueName: \"kubernetes.io/projected/abf30d8d-9a93-4298-9ee3-92887e638513-kube-api-access-zjq8r\") pod \"maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj\" (UID: \"abf30d8d-9a93-4298-9ee3-92887e638513\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:38:38.637997 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:38.637955    4521 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 03:38:38.638198 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:38.638018    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr" podUID="a8901771-3083-4017-93f8-331b57fe7b94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 03:38:38.647172 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:38.647144    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf30d8d-9a93-4298-9ee3-92887e638513-maas-api-tls\") pod \"maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj\" (UID: \"abf30d8d-9a93-4298-9ee3-92887e638513\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:38:38.649725 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:38.649689    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf30d8d-9a93-4298-9ee3-92887e638513-maas-api-tls\") pod \"maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj\" (UID: \"abf30d8d-9a93-4298-9ee3-92887e638513\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:38:38.775861 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:38.775815    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:38:39.185185 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:39.185157    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"]
Nov 04 03:38:39.186975 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:38:39.186950    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabf30d8d_9a93_4298_9ee3_92887e638513.slice/crio-1f29027130477ab0304be8dc79de580ed38b799569b7854071e6f66b8dec5444 WatchSource:0}: Error finding container 1f29027130477ab0304be8dc79de580ed38b799569b7854071e6f66b8dec5444: Status 404 returned error can't find the container with id 1f29027130477ab0304be8dc79de580ed38b799569b7854071e6f66b8dec5444
Nov 04 03:38:39.637736 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:39.637703    4521 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 03:38:39.637929 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:39.637759    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr" podUID="a8901771-3083-4017-93f8-331b57fe7b94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 03:38:39.981526 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:39.981432    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj" event={"ID":"abf30d8d-9a93-4298-9ee3-92887e638513","Type":"ContainerStarted","Data":"1f29027130477ab0304be8dc79de580ed38b799569b7854071e6f66b8dec5444"}
Nov 04 03:38:40.641577 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:40.641547    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:40.642065 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:40.642043    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:40.642612 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:40.642592    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:38:40.991310 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:40.991221    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj" event={"ID":"abf30d8d-9a93-4298-9ee3-92887e638513","Type":"ContainerStarted","Data":"a127b528f942d653001126063ff963026325c358fea02e95d9511bea84a12fd1"}
Nov 04 03:38:40.991457 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:40.991311    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:38:41.113346 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:41.110934    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj" podStartSLOduration=3.210579141 podStartE2EDuration="4.110915689s" totalImagesPullingTime="900.336548ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:38:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:38:41.109596514 +0000 UTC m=+1287.314196844" watchObservedRunningTime="2026-11-04 03:38:41.110915689 +0000 UTC m=+1287.315515992"
Nov 04 03:38:47.002136 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:38:47.002108    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:38:58.738713 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:38:58.738618    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:39:07.283232 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.283191    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"]
Nov 04 03:39:07.291597 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.291570    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.398272 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.398235    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"]
Nov 04 03:39:07.399347 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.399218    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a66091b-11c6-4dcb-8870-0ededd2187d6-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.399509 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.399390    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a66091b-11c6-4dcb-8870-0ededd2187d6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.399509 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.399416    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a66091b-11c6-4dcb-8870-0ededd2187d6-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.399673 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.399560    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zshz\" (UniqueName: \"kubernetes.io/projected/4a66091b-11c6-4dcb-8870-0ededd2187d6-kube-api-access-7zshz\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.399673 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.399614    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a66091b-11c6-4dcb-8870-0ededd2187d6-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.399776 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.399689    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a66091b-11c6-4dcb-8870-0ededd2187d6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.399776 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.399735    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a66091b-11c6-4dcb-8870-0ededd2187d6-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.399776 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.399772    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a66091b-11c6-4dcb-8870-0ededd2187d6-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.399972 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.399808    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a66091b-11c6-4dcb-8870-0ededd2187d6-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.501213 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.501174    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a66091b-11c6-4dcb-8870-0ededd2187d6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.501427 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.501232    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a66091b-11c6-4dcb-8870-0ededd2187d6-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.501427 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.501256    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a66091b-11c6-4dcb-8870-0ededd2187d6-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.501427 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.501279    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a66091b-11c6-4dcb-8870-0ededd2187d6-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.501427 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.501350    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a66091b-11c6-4dcb-8870-0ededd2187d6-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.501427 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.501381    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a66091b-11c6-4dcb-8870-0ededd2187d6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.501427 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.501398    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a66091b-11c6-4dcb-8870-0ededd2187d6-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.501772 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.501469    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zshz\" (UniqueName: \"kubernetes.io/projected/4a66091b-11c6-4dcb-8870-0ededd2187d6-kube-api-access-7zshz\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.501772 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.501519    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a66091b-11c6-4dcb-8870-0ededd2187d6-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.502048 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.502013    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a66091b-11c6-4dcb-8870-0ededd2187d6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.502048 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.502024    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a66091b-11c6-4dcb-8870-0ededd2187d6-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.502266 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.502073    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a66091b-11c6-4dcb-8870-0ededd2187d6-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.502266 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.502095    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a66091b-11c6-4dcb-8870-0ededd2187d6-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.502266 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.502155    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a66091b-11c6-4dcb-8870-0ededd2187d6-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.503970 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.503944    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a66091b-11c6-4dcb-8870-0ededd2187d6-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.504236 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.504212    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a66091b-11c6-4dcb-8870-0ededd2187d6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.562989 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.562912    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a66091b-11c6-4dcb-8870-0ededd2187d6-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.563120 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.563051    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zshz\" (UniqueName: \"kubernetes.io/projected/4a66091b-11c6-4dcb-8870-0ededd2187d6-kube-api-access-7zshz\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fms7t\" (UID: \"4a66091b-11c6-4dcb-8870-0ededd2187d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:07.605799 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:07.605764    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:08.037622 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:08.037595    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"]
Nov 04 03:39:08.040582 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:39:08.040555    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a66091b_11c6_4dcb_8870_0ededd2187d6.slice/crio-3399004f61f2c79b20874c0f7cf11d241b22f2fd7d2f2dc06b5e2dddb4479288 WatchSource:0}: Error finding container 3399004f61f2c79b20874c0f7cf11d241b22f2fd7d2f2dc06b5e2dddb4479288: Status 404 returned error can't find the container with id 3399004f61f2c79b20874c0f7cf11d241b22f2fd7d2f2dc06b5e2dddb4479288
Nov 04 03:39:08.046654 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:08.046618    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:39:08.046726 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:08.046687    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:39:08.046726 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:08.046719    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:39:08.151349 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:08.151312    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t" event={"ID":"4a66091b-11c6-4dcb-8870-0ededd2187d6","Type":"ContainerStarted","Data":"68f8b948e7baf7c9e99468fa8117bace95a619b8af5f7c18f5ee7ab6bf189e48"}
Nov 04 03:39:08.151526 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:08.151376    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t" event={"ID":"4a66091b-11c6-4dcb-8870-0ededd2187d6","Type":"ContainerStarted","Data":"3399004f61f2c79b20874c0f7cf11d241b22f2fd7d2f2dc06b5e2dddb4479288"}
Nov 04 03:39:08.272599 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:08.272547    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t" podStartSLOduration=1.272526466 podStartE2EDuration="1.272526466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:39:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:39:08.271866806 +0000 UTC m=+1314.476467144" watchObservedRunningTime="2026-11-04 03:39:08.272526466 +0000 UTC m=+1314.477126773"
Nov 04 03:39:08.606435 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:08.606402    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:09.609900 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:09.609825    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fms7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 03:39:09.610532 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:09.610482    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t" podUID="4a66091b-11c6-4dcb-8870-0ededd2187d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 03:39:10.606311 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:10.606267    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fms7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 03:39:10.606595 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:10.606560    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t" podUID="4a66091b-11c6-4dcb-8870-0ededd2187d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 03:39:11.606867 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:11.606253    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fms7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 03:39:11.606867 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:11.606320    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t" podUID="4a66091b-11c6-4dcb-8870-0ededd2187d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 03:39:12.606281 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:12.606240    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fms7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 03:39:12.606608 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:12.606575    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t" podUID="4a66091b-11c6-4dcb-8870-0ededd2187d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 03:39:13.609870 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:13.608200    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fms7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 03:39:13.609870 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:13.608263    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t" podUID="4a66091b-11c6-4dcb-8870-0ededd2187d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 03:39:14.606477 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:14.606440    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fms7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 03:39:14.606667 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:14.606495    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t" podUID="4a66091b-11c6-4dcb-8870-0ededd2187d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 03:39:15.606405 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:15.606363    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fms7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 03:39:15.606797 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:15.606424    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t" podUID="4a66091b-11c6-4dcb-8870-0ededd2187d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 03:39:16.827572 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:16.827536    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:16.828226 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:16.827964    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:16.829007 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:16.828984    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fms7t"
Nov 04 03:39:22.400568 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.400530    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"]
Nov 04 03:39:22.406196 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.406168    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.484549 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.484510    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a71c9284-671b-4ed4-8aac-7687a2e3eefe-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.484716 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.484565    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a71c9284-671b-4ed4-8aac-7687a2e3eefe-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.484716 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.484677    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a71c9284-671b-4ed4-8aac-7687a2e3eefe-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.484787 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.484718    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a71c9284-671b-4ed4-8aac-7687a2e3eefe-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.484912 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.484887    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a71c9284-671b-4ed4-8aac-7687a2e3eefe-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.484960 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.484945    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a71c9284-671b-4ed4-8aac-7687a2e3eefe-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.485050 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.485030    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4nqt\" (UniqueName: \"kubernetes.io/projected/a71c9284-671b-4ed4-8aac-7687a2e3eefe-kube-api-access-c4nqt\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.485085 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.485076    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a71c9284-671b-4ed4-8aac-7687a2e3eefe-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.485118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.485104    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a71c9284-671b-4ed4-8aac-7687a2e3eefe-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.513201 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.513169    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"]
Nov 04 03:39:22.586241 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.586207    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a71c9284-671b-4ed4-8aac-7687a2e3eefe-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.586436 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.586259    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a71c9284-671b-4ed4-8aac-7687a2e3eefe-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.586436 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.586291    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4nqt\" (UniqueName: \"kubernetes.io/projected/a71c9284-671b-4ed4-8aac-7687a2e3eefe-kube-api-access-c4nqt\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.586436 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.586314    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a71c9284-671b-4ed4-8aac-7687a2e3eefe-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.586436 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.586331    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a71c9284-671b-4ed4-8aac-7687a2e3eefe-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.586662 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.586554    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a71c9284-671b-4ed4-8aac-7687a2e3eefe-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.586662 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.586601    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a71c9284-671b-4ed4-8aac-7687a2e3eefe-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.586662 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.586615    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a71c9284-671b-4ed4-8aac-7687a2e3eefe-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.586789 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.586675    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a71c9284-671b-4ed4-8aac-7687a2e3eefe-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.586789 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.586693    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a71c9284-671b-4ed4-8aac-7687a2e3eefe-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.586789 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.586741    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a71c9284-671b-4ed4-8aac-7687a2e3eefe-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.586982 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.586946    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a71c9284-671b-4ed4-8aac-7687a2e3eefe-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.587063 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.587042    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a71c9284-671b-4ed4-8aac-7687a2e3eefe-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.587191 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.587167    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a71c9284-671b-4ed4-8aac-7687a2e3eefe-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.588743 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.588709    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a71c9284-671b-4ed4-8aac-7687a2e3eefe-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.588933 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.588915    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a71c9284-671b-4ed4-8aac-7687a2e3eefe-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.647897 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.647837    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a71c9284-671b-4ed4-8aac-7687a2e3eefe-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.648072 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.647958    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4nqt\" (UniqueName: \"kubernetes.io/projected/a71c9284-671b-4ed4-8aac-7687a2e3eefe-kube-api-access-c4nqt\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-sr9mv\" (UID: \"a71c9284-671b-4ed4-8aac-7687a2e3eefe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:22.720345 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:22.720263    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:23.150810 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:23.150778    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"]
Nov 04 03:39:23.152462 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:39:23.152437    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda71c9284_671b_4ed4_8aac_7687a2e3eefe.slice/crio-8118384d0731e0c2339ac6ebca4fc26b027bce65296cf1276f3be05500d52d72 WatchSource:0}: Error finding container 8118384d0731e0c2339ac6ebca4fc26b027bce65296cf1276f3be05500d52d72: Status 404 returned error can't find the container with id 8118384d0731e0c2339ac6ebca4fc26b027bce65296cf1276f3be05500d52d72
Nov 04 03:39:23.154906 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:23.154832    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:39:23.155000 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:23.154939    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:39:23.155000 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:23.154968    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:39:23.247404 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:23.247352    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv" event={"ID":"a71c9284-671b-4ed4-8aac-7687a2e3eefe","Type":"ContainerStarted","Data":"5dd163e32df10fd2eff4e6c99e04efd748481118dd6e82e256ff099a549238cc"}
Nov 04 03:39:23.247581 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:23.247428    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv" event={"ID":"a71c9284-671b-4ed4-8aac-7687a2e3eefe","Type":"ContainerStarted","Data":"8118384d0731e0c2339ac6ebca4fc26b027bce65296cf1276f3be05500d52d72"}
Nov 04 03:39:23.373310 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:23.373245    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv" podStartSLOduration=1.373224554 podStartE2EDuration="1.373224554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:39:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:39:23.368640278 +0000 UTC m=+1329.573240600" watchObservedRunningTime="2026-11-04 03:39:23.373224554 +0000 UTC m=+1329.577824861"
Nov 04 03:39:23.720901 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:23.720866    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:24.722543 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:24.722506    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 04 03:39:24.722990 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:24.722570    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv" podUID="a71c9284-671b-4ed4-8aac-7687a2e3eefe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 04 03:39:25.721537 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:25.721498    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 04 03:39:25.721728 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:25.721569    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv" podUID="a71c9284-671b-4ed4-8aac-7687a2e3eefe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 04 03:39:26.721149 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:26.721115    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 04 03:39:26.721573 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:26.721175    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv" podUID="a71c9284-671b-4ed4-8aac-7687a2e3eefe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 04 03:39:27.721350 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:27.721303    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 04 03:39:27.721744 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:27.721364    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv" podUID="a71c9284-671b-4ed4-8aac-7687a2e3eefe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 04 03:39:28.536216 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:28.536178    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"]
Nov 04 03:39:28.610834 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:28.610794    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"]
Nov 04 03:39:28.613614 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:28.613580    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"]
Nov 04 03:39:28.721545 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:28.721505    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 04 03:39:28.721949 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:28.721572    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv" podUID="a71c9284-671b-4ed4-8aac-7687a2e3eefe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 04 03:39:28.738399 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:39:28.738337    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:39:29.720670 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:29.720626    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 04 03:39:29.720908 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:29.720693    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv" podUID="a71c9284-671b-4ed4-8aac-7687a2e3eefe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 04 03:39:30.721374 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:30.721331    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 04 03:39:30.721772 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:30.721386    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv" podUID="a71c9284-671b-4ed4-8aac-7687a2e3eefe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 04 03:39:32.109074 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:32.109035    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:32.110055 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:32.109990    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:32.112753 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:32.112725    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-sr9mv"
Nov 04 03:39:40.386867 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:40.385981    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b" podUID="672c902d-f63e-42e6-b1e2-407d6e148c04" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:39:40.449508 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:40.449468    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"]
Nov 04 03:39:40.479468 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:40.479421    4521 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b" podUID="672c902d-f63e-42e6-b1e2-407d6e148c04" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": context deadline exceeded"
Nov 04 03:39:40.507869 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:40.482801    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"
Nov 04 03:39:40.539770 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:40.539717    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc" podUID="b8ab2132-5f2b-43ef-8a57-acfa16dc8090" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:39:40.570453 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:40.570419    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"]
Nov 04 03:39:40.619722 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:40.619688    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-951893-dockercfg-6gdx2\""
Nov 04 03:39:40.620046 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:40.620024    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-951893\""
Nov 04 03:39:40.682576 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:40.682468    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h24t\" (UniqueName: \"kubernetes.io/projected/8676d045-b8b9-4939-bd4b-089b0d0df9ee-kube-api-access-4h24t\") pod \"payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x\" (UID: \"8676d045-b8b9-4939-bd4b-089b0d0df9ee\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"
Nov 04 03:39:40.683022 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:40.682995    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8676d045-b8b9-4939-bd4b-089b0d0df9ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x\" (UID: \"8676d045-b8b9-4939-bd4b-089b0d0df9ee\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"
Nov 04 03:39:40.786888 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:40.786828    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h24t\" (UniqueName: \"kubernetes.io/projected/8676d045-b8b9-4939-bd4b-089b0d0df9ee-kube-api-access-4h24t\") pod \"payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x\" (UID: \"8676d045-b8b9-4939-bd4b-089b0d0df9ee\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"
Nov 04 03:39:40.787280 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:40.787242    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8676d045-b8b9-4939-bd4b-089b0d0df9ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x\" (UID: \"8676d045-b8b9-4939-bd4b-089b0d0df9ee\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"
Nov 04 03:39:40.789221 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:40.789199    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8676d045-b8b9-4939-bd4b-089b0d0df9ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x\" (UID: \"8676d045-b8b9-4939-bd4b-089b0d0df9ee\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"
Nov 04 03:39:40.886687 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:40.886650    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h24t\" (UniqueName: \"kubernetes.io/projected/8676d045-b8b9-4939-bd4b-089b0d0df9ee-kube-api-access-4h24t\") pod \"payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x\" (UID: \"8676d045-b8b9-4939-bd4b-089b0d0df9ee\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"
Nov 04 03:39:40.913867 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:40.902934    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7" podUID="5932542b-7aff-428d-a8e9-b8a20235035d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:39:40.937923 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:40.937090    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"
Nov 04 03:39:41.229932 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:41.220381    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj" podUID="29ed832f-92f7-4a13-9a94-b65f05ab260c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 04 03:39:41.272370 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:41.271971    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt" podUID="b3eb20c6-1760-4310-9edf-7b6392b89b63" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:39:41.673109 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:41.672020    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"]
Nov 04 03:39:42.617476 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:42.617444    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x" event={"ID":"8676d045-b8b9-4939-bd4b-089b0d0df9ee","Type":"ContainerStarted","Data":"f1a61f6fcd2b4e982b323a2e6c2acc8eaf0030a0dc9bc3b788b8e66e312d6b2d"}
Nov 04 03:39:42.617696 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:42.617498    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x" event={"ID":"8676d045-b8b9-4939-bd4b-089b0d0df9ee","Type":"ContainerStarted","Data":"17416f9ab9624855ac869538db409902edb3d1c4ac53796066797b9733af7e4a"}
Nov 04 03:39:42.617696 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:42.617525    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"
Nov 04 03:39:42.736555 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:42.736495    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x" podStartSLOduration=2.736480694 podStartE2EDuration="2.736480694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:39:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:39:42.733661348 +0000 UTC m=+1348.938261650" watchObservedRunningTime="2026-11-04 03:39:42.736480694 +0000 UTC m=+1348.941080997"
Nov 04 03:39:43.560726 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:43.560685    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc" podUID="b8ab2132-5f2b-43ef-8a57-acfa16dc8090" containerName="main" containerID="cri-o://e9968b85da209b9578bc2f2b02b9ff092dcfeaf34e462a2ed840a3c899777e91" gracePeriod=45
Nov 04 03:39:43.601447 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:43.601412    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"]
Nov 04 03:39:43.631957 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:43.631913    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj" podUID="29ed832f-92f7-4a13-9a94-b65f05ab260c" containerName="main" containerID="cri-o://bde1f3573a464a2d2c8bf4a5e0c0f7eafce9b00756b0641e5d1d0bb56d57856c" gracePeriod=45
Nov 04 03:39:43.634033 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:43.633980    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt" podUID="b3eb20c6-1760-4310-9edf-7b6392b89b63" containerName="main" containerID="cri-o://d3a8a9a973571e44c16a5c1f8d4d409598b3d318d8118aa3d5a391198c87c71d" gracePeriod=45
Nov 04 03:39:43.998932 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:43.998896    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:39:44.003920 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.003894    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:39:44.008596 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.008571    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:39:44.040800 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.040770    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-dshm\") pod \"29ed832f-92f7-4a13-9a94-b65f05ab260c\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") "
Nov 04 03:39:44.040991 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.040824    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-model-cache\") pod \"b3eb20c6-1760-4310-9edf-7b6392b89b63\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") "
Nov 04 03:39:44.040991 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.040865    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-home\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-home\") pod \"b3eb20c6-1760-4310-9edf-7b6392b89b63\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") "
Nov 04 03:39:44.040991 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.040913    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-kube-api-access-r4sbj\" (UniqueName: \"kubernetes.io/projected/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-kube-api-access-r4sbj\") pod \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") "
Nov 04 03:39:44.040991 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.040975    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3eb20c6-1760-4310-9edf-7b6392b89b63-kube-api-access-2zjf8\" (UniqueName: \"kubernetes.io/projected/b3eb20c6-1760-4310-9edf-7b6392b89b63-kube-api-access-2zjf8\") pod \"b3eb20c6-1760-4310-9edf-7b6392b89b63\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") "
Nov 04 03:39:44.041214 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041007    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-model-cache\") pod \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") "
Nov 04 03:39:44.041214 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041030    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-model-cache\") pod \"29ed832f-92f7-4a13-9a94-b65f05ab260c\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") "
Nov 04 03:39:44.041214 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041111    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/29ed832f-92f7-4a13-9a94-b65f05ab260c-tls-certs\" (UniqueName: \"kubernetes.io/secret/29ed832f-92f7-4a13-9a94-b65f05ab260c-tls-certs\") pod \"29ed832f-92f7-4a13-9a94-b65f05ab260c\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") "
Nov 04 03:39:44.041214 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041171    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-tls-certs\") pod \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") "
Nov 04 03:39:44.041420 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041209    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29ed832f-92f7-4a13-9a94-b65f05ab260c-kube-api-access-tqfrn\" (UniqueName: \"kubernetes.io/projected/29ed832f-92f7-4a13-9a94-b65f05ab260c-kube-api-access-tqfrn\") pod \"29ed832f-92f7-4a13-9a94-b65f05ab260c\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") "
Nov 04 03:39:44.041420 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041314    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-tmp-dir\") pod \"b3eb20c6-1760-4310-9edf-7b6392b89b63\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") "
Nov 04 03:39:44.041420 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041118    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-home" pod "b3eb20c6-1760-4310-9edf-7b6392b89b63" (UID: "b3eb20c6-1760-4310-9edf-7b6392b89b63"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:39:44.041420 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041272    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-model-cache" pod "b3eb20c6-1760-4310-9edf-7b6392b89b63" (UID: "b3eb20c6-1760-4310-9edf-7b6392b89b63"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:39:44.041420 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041278    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-model-cache" pod "29ed832f-92f7-4a13-9a94-b65f05ab260c" (UID: "29ed832f-92f7-4a13-9a94-b65f05ab260c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:39:44.041420 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041313    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-model-cache" pod "b8ab2132-5f2b-43ef-8a57-acfa16dc8090" (UID: "b8ab2132-5f2b-43ef-8a57-acfa16dc8090"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:39:44.041420 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041409    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-home\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-home\") pod \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") "
Nov 04 03:39:44.041791 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041450    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-dshm\") pod \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") "
Nov 04 03:39:44.041791 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041493    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-tmp-dir\") pod \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\" (UID: \"b8ab2132-5f2b-43ef-8a57-acfa16dc8090\") "
Nov 04 03:39:44.041791 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041539    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-tmp-dir\") pod \"29ed832f-92f7-4a13-9a94-b65f05ab260c\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") "
Nov 04 03:39:44.041791 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041559    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-tmp-dir" pod "b3eb20c6-1760-4310-9edf-7b6392b89b63" (UID: "b3eb20c6-1760-4310-9edf-7b6392b89b63"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:39:44.041791 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041569    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-dshm\") pod \"b3eb20c6-1760-4310-9edf-7b6392b89b63\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") "
Nov 04 03:39:44.041791 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041612    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b3eb20c6-1760-4310-9edf-7b6392b89b63-tls-certs\" (UniqueName: \"kubernetes.io/secret/b3eb20c6-1760-4310-9edf-7b6392b89b63-tls-certs\") pod \"b3eb20c6-1760-4310-9edf-7b6392b89b63\" (UID: \"b3eb20c6-1760-4310-9edf-7b6392b89b63\") "
Nov 04 03:39:44.041791 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.041632    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-home\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-home\") pod \"29ed832f-92f7-4a13-9a94-b65f05ab260c\" (UID: \"29ed832f-92f7-4a13-9a94-b65f05ab260c\") "
Nov 04 03:39:44.042425 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.042399    4521 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-model-cache\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.042512 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.042429    4521 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-home\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.042512 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.042446    4521 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-model-cache\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.042512 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.042461    4521 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-model-cache\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.042512 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.042477    4521 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-tmp-dir\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.042734 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.042697    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-home" pod "29ed832f-92f7-4a13-9a94-b65f05ab260c" (UID: "29ed832f-92f7-4a13-9a94-b65f05ab260c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:39:44.042914 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.042891    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-tmp-dir" pod "b8ab2132-5f2b-43ef-8a57-acfa16dc8090" (UID: "b8ab2132-5f2b-43ef-8a57-acfa16dc8090"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:39:44.043104 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.043084    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-tmp-dir" pod "29ed832f-92f7-4a13-9a94-b65f05ab260c" (UID: "29ed832f-92f7-4a13-9a94-b65f05ab260c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:39:44.044019 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.043722    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-dshm" pod "29ed832f-92f7-4a13-9a94-b65f05ab260c" (UID: "29ed832f-92f7-4a13-9a94-b65f05ab260c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:39:44.044019 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.043723    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-kube-api-access-r4sbj" pod "b8ab2132-5f2b-43ef-8a57-acfa16dc8090" (UID: "b8ab2132-5f2b-43ef-8a57-acfa16dc8090"). InnerVolumeSpecName "kube-api-access-r4sbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:39:44.044019 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.043902    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-tls-certs" pod "b8ab2132-5f2b-43ef-8a57-acfa16dc8090" (UID: "b8ab2132-5f2b-43ef-8a57-acfa16dc8090"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:39:44.044019 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.043988    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3eb20c6-1760-4310-9edf-7b6392b89b63-kube-api-access-2zjf8" pod "b3eb20c6-1760-4310-9edf-7b6392b89b63" (UID: "b3eb20c6-1760-4310-9edf-7b6392b89b63"). InnerVolumeSpecName "kube-api-access-2zjf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:39:44.044019 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.043995    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-home" pod "b8ab2132-5f2b-43ef-8a57-acfa16dc8090" (UID: "b8ab2132-5f2b-43ef-8a57-acfa16dc8090"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:39:44.044019 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.044003    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-dshm" pod "b8ab2132-5f2b-43ef-8a57-acfa16dc8090" (UID: "b8ab2132-5f2b-43ef-8a57-acfa16dc8090"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:39:44.044301 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.044174    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29ed832f-92f7-4a13-9a94-b65f05ab260c-tls-certs" pod "29ed832f-92f7-4a13-9a94-b65f05ab260c" (UID: "29ed832f-92f7-4a13-9a94-b65f05ab260c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:39:44.045081 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.045059    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-dshm" pod "b3eb20c6-1760-4310-9edf-7b6392b89b63" (UID: "b3eb20c6-1760-4310-9edf-7b6392b89b63"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:39:44.045244 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.045223    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29ed832f-92f7-4a13-9a94-b65f05ab260c-kube-api-access-tqfrn" pod "29ed832f-92f7-4a13-9a94-b65f05ab260c" (UID: "29ed832f-92f7-4a13-9a94-b65f05ab260c"). InnerVolumeSpecName "kube-api-access-tqfrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:39:44.045548 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.045531    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3eb20c6-1760-4310-9edf-7b6392b89b63-tls-certs" pod "b3eb20c6-1760-4310-9edf-7b6392b89b63" (UID: "b3eb20c6-1760-4310-9edf-7b6392b89b63"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:39:44.152239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.145375    4521 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-dshm\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.152239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.145420    4521 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-tmp-dir\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.152239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.145436    4521 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-tmp-dir\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.152239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.145450    4521 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3eb20c6-1760-4310-9edf-7b6392b89b63-dshm\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.152239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.145464    4521 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3eb20c6-1760-4310-9edf-7b6392b89b63-tls-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.152239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.145483    4521 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-home\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.152239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.145496    4521 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29ed832f-92f7-4a13-9a94-b65f05ab260c-dshm\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.152239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.145510    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4sbj\" (UniqueName: \"kubernetes.io/projected/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-kube-api-access-r4sbj\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.152239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.145527    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zjf8\" (UniqueName: \"kubernetes.io/projected/b3eb20c6-1760-4310-9edf-7b6392b89b63-kube-api-access-2zjf8\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.152239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.145545    4521 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29ed832f-92f7-4a13-9a94-b65f05ab260c-tls-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.152239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.145559    4521 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-tls-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.152239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.145572    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqfrn\" (UniqueName: \"kubernetes.io/projected/29ed832f-92f7-4a13-9a94-b65f05ab260c-kube-api-access-tqfrn\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.152239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.145586    4521 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8ab2132-5f2b-43ef-8a57-acfa16dc8090-home\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.633889 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.633855    4521 generic.go:356] "Generic (PLEG): container finished" podID="b3eb20c6-1760-4310-9edf-7b6392b89b63" containerID="d3a8a9a973571e44c16a5c1f8d4d409598b3d318d8118aa3d5a391198c87c71d" exitCode=0
Nov 04 03:39:44.634110 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.633917    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"
Nov 04 03:39:44.634110 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.633932    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt" event={"ID":"b3eb20c6-1760-4310-9edf-7b6392b89b63","Type":"ContainerDied","Data":"d3a8a9a973571e44c16a5c1f8d4d409598b3d318d8118aa3d5a391198c87c71d"}
Nov 04 03:39:44.634110 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.633981    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt" event={"ID":"b3eb20c6-1760-4310-9edf-7b6392b89b63","Type":"ContainerDied","Data":"e1060eca4885d70555814f142623dda05cabfd9a97d3f173e5c0ccdda2f08300"}
Nov 04 03:39:44.634110 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.634005    4521 scope.go:122] "RemoveContainer" containerID="d3a8a9a973571e44c16a5c1f8d4d409598b3d318d8118aa3d5a391198c87c71d"
Nov 04 03:39:44.635420 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.635378    4521 generic.go:356] "Generic (PLEG): container finished" podID="29ed832f-92f7-4a13-9a94-b65f05ab260c" containerID="bde1f3573a464a2d2c8bf4a5e0c0f7eafce9b00756b0641e5d1d0bb56d57856c" exitCode=0
Nov 04 03:39:44.635518 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.635453    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"
Nov 04 03:39:44.635518 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.635495    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj" event={"ID":"29ed832f-92f7-4a13-9a94-b65f05ab260c","Type":"ContainerDied","Data":"bde1f3573a464a2d2c8bf4a5e0c0f7eafce9b00756b0641e5d1d0bb56d57856c"}
Nov 04 03:39:44.635624 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.635527    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj" event={"ID":"29ed832f-92f7-4a13-9a94-b65f05ab260c","Type":"ContainerDied","Data":"b92ec904227962763701b2cf7153eab67debced72fd6c06a8c8f1570aa529ae2"}
Nov 04 03:39:44.637054 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.637031    4521 generic.go:356] "Generic (PLEG): container finished" podID="b8ab2132-5f2b-43ef-8a57-acfa16dc8090" containerID="e9968b85da209b9578bc2f2b02b9ff092dcfeaf34e462a2ed840a3c899777e91" exitCode=0
Nov 04 03:39:44.637163 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.637087    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"
Nov 04 03:39:44.637163 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.637097    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc" event={"ID":"b8ab2132-5f2b-43ef-8a57-acfa16dc8090","Type":"ContainerDied","Data":"e9968b85da209b9578bc2f2b02b9ff092dcfeaf34e462a2ed840a3c899777e91"}
Nov 04 03:39:44.637282 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.637165    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc" event={"ID":"b8ab2132-5f2b-43ef-8a57-acfa16dc8090","Type":"ContainerDied","Data":"2630be14dba77a4be05e37ab18fa7c4807d2bf360158ec58f575eba644724499"}
Nov 04 03:39:44.644064 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.644041    4521 scope.go:122] "RemoveContainer" containerID="d3a8a9a973571e44c16a5c1f8d4d409598b3d318d8118aa3d5a391198c87c71d"
Nov 04 03:39:44.644312 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:39:44.644287    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3a8a9a973571e44c16a5c1f8d4d409598b3d318d8118aa3d5a391198c87c71d\": container with ID starting with d3a8a9a973571e44c16a5c1f8d4d409598b3d318d8118aa3d5a391198c87c71d not found: ID does not exist" containerID="d3a8a9a973571e44c16a5c1f8d4d409598b3d318d8118aa3d5a391198c87c71d"
Nov 04 03:39:44.644312 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.644313    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a8a9a973571e44c16a5c1f8d4d409598b3d318d8118aa3d5a391198c87c71d"} err="failed to get container status \"d3a8a9a973571e44c16a5c1f8d4d409598b3d318d8118aa3d5a391198c87c71d\": rpc error: code = NotFound desc = could not find container \"d3a8a9a973571e44c16a5c1f8d4d409598b3d318d8118aa3d5a391198c87c71d\": container with ID starting with d3a8a9a973571e44c16a5c1f8d4d409598b3d318d8118aa3d5a391198c87c71d not found: ID does not exist"
Nov 04 03:39:44.644497 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.644334    4521 scope.go:122] "RemoveContainer" containerID="bde1f3573a464a2d2c8bf4a5e0c0f7eafce9b00756b0641e5d1d0bb56d57856c"
Nov 04 03:39:44.654133 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.654117    4521 scope.go:122] "RemoveContainer" containerID="bde1f3573a464a2d2c8bf4a5e0c0f7eafce9b00756b0641e5d1d0bb56d57856c"
Nov 04 03:39:44.654362 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:39:44.654345    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bde1f3573a464a2d2c8bf4a5e0c0f7eafce9b00756b0641e5d1d0bb56d57856c\": container with ID starting with bde1f3573a464a2d2c8bf4a5e0c0f7eafce9b00756b0641e5d1d0bb56d57856c not found: ID does not exist" containerID="bde1f3573a464a2d2c8bf4a5e0c0f7eafce9b00756b0641e5d1d0bb56d57856c"
Nov 04 03:39:44.654421 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.654376    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bde1f3573a464a2d2c8bf4a5e0c0f7eafce9b00756b0641e5d1d0bb56d57856c"} err="failed to get container status \"bde1f3573a464a2d2c8bf4a5e0c0f7eafce9b00756b0641e5d1d0bb56d57856c\": rpc error: code = NotFound desc = could not find container \"bde1f3573a464a2d2c8bf4a5e0c0f7eafce9b00756b0641e5d1d0bb56d57856c\": container with ID starting with bde1f3573a464a2d2c8bf4a5e0c0f7eafce9b00756b0641e5d1d0bb56d57856c not found: ID does not exist"
Nov 04 03:39:44.654421 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.654398    4521 scope.go:122] "RemoveContainer" containerID="e9968b85da209b9578bc2f2b02b9ff092dcfeaf34e462a2ed840a3c899777e91"
Nov 04 03:39:44.663099 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.663085    4521 scope.go:122] "RemoveContainer" containerID="e9968b85da209b9578bc2f2b02b9ff092dcfeaf34e462a2ed840a3c899777e91"
Nov 04 03:39:44.663363 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:39:44.663346    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9968b85da209b9578bc2f2b02b9ff092dcfeaf34e462a2ed840a3c899777e91\": container with ID starting with e9968b85da209b9578bc2f2b02b9ff092dcfeaf34e462a2ed840a3c899777e91 not found: ID does not exist" containerID="e9968b85da209b9578bc2f2b02b9ff092dcfeaf34e462a2ed840a3c899777e91"
Nov 04 03:39:44.663413 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.663368    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9968b85da209b9578bc2f2b02b9ff092dcfeaf34e462a2ed840a3c899777e91"} err="failed to get container status \"e9968b85da209b9578bc2f2b02b9ff092dcfeaf34e462a2ed840a3c899777e91\": rpc error: code = NotFound desc = could not find container \"e9968b85da209b9578bc2f2b02b9ff092dcfeaf34e462a2ed840a3c899777e91\": container with ID starting with e9968b85da209b9578bc2f2b02b9ff092dcfeaf34e462a2ed840a3c899777e91 not found: ID does not exist"
Nov 04 03:39:44.801088 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.801048    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"]
Nov 04 03:39:44.805107 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.805077    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/premium-w1-ee1947-kserve-7cffd84d9d-2sdvt"]
Nov 04 03:39:44.917189 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.917079    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"]
Nov 04 03:39:44.921995 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:44.921967    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/distinct-2-w1-ee1947-kserve-665fb85566-ds8pc"]
Nov 04 03:39:45.034348 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:45.034312    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"]
Nov 04 03:39:45.041818 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:45.041768    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/distinct-w1-ee1947-kserve-5dc45d786f-kcpqj"]
Nov 04 03:39:45.912377 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:45.912333    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29ed832f-92f7-4a13-9a94-b65f05ab260c" path="/var/lib/kubelet/pods/29ed832f-92f7-4a13-9a94-b65f05ab260c/volumes"
Nov 04 03:39:45.912924 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:45.912901    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3eb20c6-1760-4310-9edf-7b6392b89b63" path="/var/lib/kubelet/pods/b3eb20c6-1760-4310-9edf-7b6392b89b63/volumes"
Nov 04 03:39:45.913385 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:45.913363    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8ab2132-5f2b-43ef-8a57-acfa16dc8090" path="/var/lib/kubelet/pods/b8ab2132-5f2b-43ef-8a57-acfa16dc8090/volumes"
Nov 04 03:39:48.601740 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:48.601688    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq" podUID="1b7602b8-2d7b-42b4-acd3-868233206892" containerName="payload-pre-processing" containerID="cri-o://6c97d4c14e24737ac15ca296dcd21fc061e0ec1c653e4ef8f99048d8436655a3" gracePeriod=25
Nov 04 03:39:50.417570 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:50.417525    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq" podUID="1b7602b8-2d7b-42b4-acd3-868233206892" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:39:53.630435 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:39:53.630397    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"
Nov 04 03:39:58.734873 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:39:58.734817    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:40:00.417192 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:00.417144    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq" podUID="1b7602b8-2d7b-42b4-acd3-868233206892" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:40:02.537796 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.537757    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"]
Nov 04 03:40:02.538437 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.538417    4521 cpu_manager.go:414] "containerMap: removing container" podUID="b8ab2132-5f2b-43ef-8a57-acfa16dc8090" containerName="main"
Nov 04 03:40:02.538483 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.538438    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8ab2132-5f2b-43ef-8a57-acfa16dc8090" containerName="main"
Nov 04 03:40:02.538483 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.538463    4521 cpu_manager.go:414] "containerMap: removing container" podUID="29ed832f-92f7-4a13-9a94-b65f05ab260c" containerName="main"
Nov 04 03:40:02.538483 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.538468    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29ed832f-92f7-4a13-9a94-b65f05ab260c" containerName="main"
Nov 04 03:40:02.538619 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.538499    4521 cpu_manager.go:414] "containerMap: removing container" podUID="b3eb20c6-1760-4310-9edf-7b6392b89b63" containerName="main"
Nov 04 03:40:02.538619 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.538508    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3eb20c6-1760-4310-9edf-7b6392b89b63" containerName="main"
Nov 04 03:40:02.538683 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.538620    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3eb20c6-1760-4310-9edf-7b6392b89b63" containerName="main"
Nov 04 03:40:02.538683 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.538629    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8ab2132-5f2b-43ef-8a57-acfa16dc8090" containerName="main"
Nov 04 03:40:02.538683 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.538637    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="29ed832f-92f7-4a13-9a94-b65f05ab260c" containerName="main"
Nov 04 03:40:02.543937 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.543911    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:40:02.599266 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.599237    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-59c311\"/\"agc-cred-7fc405a9bd4c\""
Nov 04 03:40:02.599443 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.599284    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-59c311\"/\"payload-processing-external-model-e2e--ef846cec-dockercfg-jwwvl\""
Nov 04 03:40:02.649807 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.649770    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-59c311\"/\"openshift-service-ca.crt\""
Nov 04 03:40:02.649807 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.649801    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-59c311\"/\"payload-processing-external-model-plugins-e2e-extmod-b-59c311\""
Nov 04 03:40:02.650086 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.649834    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-59c311\"/\"routing-overlay\""
Nov 04 03:40:02.650086 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.649890    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-59c311\"/\"kube-root-ca.crt\""
Nov 04 03:40:02.652889 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.652858    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"]
Nov 04 03:40:02.669073 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.669049    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x76zj\" (UniqueName: \"kubernetes.io/projected/811373cd-381a-4937-9b9c-594a56dc96d3-kube-api-access-x76zj\") pod \"payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c\" (UID: \"811373cd-381a-4937-9b9c-594a56dc96d3\") " pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:40:02.669199 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.669099    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/811373cd-381a-4937-9b9c-594a56dc96d3-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c\" (UID: \"811373cd-381a-4937-9b9c-594a56dc96d3\") " pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:40:02.669245 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.669195    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/811373cd-381a-4937-9b9c-594a56dc96d3-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c\" (UID: \"811373cd-381a-4937-9b9c-594a56dc96d3\") " pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:40:02.669312 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.669294    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/811373cd-381a-4937-9b9c-594a56dc96d3-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c\" (UID: \"811373cd-381a-4937-9b9c-594a56dc96d3\") " pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:40:02.770516 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.770482    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/811373cd-381a-4937-9b9c-594a56dc96d3-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c\" (UID: \"811373cd-381a-4937-9b9c-594a56dc96d3\") " pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:40:02.770714 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.770580    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x76zj\" (UniqueName: \"kubernetes.io/projected/811373cd-381a-4937-9b9c-594a56dc96d3-kube-api-access-x76zj\") pod \"payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c\" (UID: \"811373cd-381a-4937-9b9c-594a56dc96d3\") " pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:40:02.770714 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.770615    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/811373cd-381a-4937-9b9c-594a56dc96d3-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c\" (UID: \"811373cd-381a-4937-9b9c-594a56dc96d3\") " pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:40:02.770714 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.770668    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/811373cd-381a-4937-9b9c-594a56dc96d3-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c\" (UID: \"811373cd-381a-4937-9b9c-594a56dc96d3\") " pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:40:02.771265 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.771245    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/811373cd-381a-4937-9b9c-594a56dc96d3-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c\" (UID: \"811373cd-381a-4937-9b9c-594a56dc96d3\") " pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:40:02.771346 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.771324    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/811373cd-381a-4937-9b9c-594a56dc96d3-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c\" (UID: \"811373cd-381a-4937-9b9c-594a56dc96d3\") " pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:40:02.773215 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.773189    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/811373cd-381a-4937-9b9c-594a56dc96d3-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c\" (UID: \"811373cd-381a-4937-9b9c-594a56dc96d3\") " pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:40:02.832992 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.832923    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x76zj\" (UniqueName: \"kubernetes.io/projected/811373cd-381a-4937-9b9c-594a56dc96d3-kube-api-access-x76zj\") pod \"payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c\" (UID: \"811373cd-381a-4937-9b9c-594a56dc96d3\") " pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:40:02.858934 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:02.858899    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:40:03.238857 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:03.238814    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"]
Nov 04 03:40:03.240004 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:40:03.239982    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod811373cd_381a_4937_9b9c_594a56dc96d3.slice/crio-1c90ce5e39f22f302d23a6699aa60ae68957c3f597054d3be502b138fde96ec3 WatchSource:0}: Error finding container 1c90ce5e39f22f302d23a6699aa60ae68957c3f597054d3be502b138fde96ec3: Status 404 returned error can't find the container with id 1c90ce5e39f22f302d23a6699aa60ae68957c3f597054d3be502b138fde96ec3
Nov 04 03:40:03.761233 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:03.761200    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c" event={"ID":"811373cd-381a-4937-9b9c-594a56dc96d3","Type":"ContainerStarted","Data":"62291a5826018042d05981ee52a49db56f81daf2594973d157fa5374e14f51ca"}
Nov 04 03:40:03.761640 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:03.761256    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c" event={"ID":"811373cd-381a-4937-9b9c-594a56dc96d3","Type":"ContainerStarted","Data":"1c90ce5e39f22f302d23a6699aa60ae68957c3f597054d3be502b138fde96ec3"}
Nov 04 03:40:03.761640 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:03.761392    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:40:03.882720 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:03.882668    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c" podStartSLOduration=1.882654385 podStartE2EDuration="1.882654385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:40:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:40:03.879267934 +0000 UTC m=+1370.083868249" watchObservedRunningTime="2026-11-04 03:40:03.882654385 +0000 UTC m=+1370.087254687"
Nov 04 03:40:07.826802 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:07.826762    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"]
Nov 04 03:40:07.831102 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:07.831082    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:07.932859 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:07.932811    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba2f1b97-4d8d-4b69-bf55-531600544660-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:07.933054 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:07.932885    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba2f1b97-4d8d-4b69-bf55-531600544660-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:07.933054 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:07.933036    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba2f1b97-4d8d-4b69-bf55-531600544660-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:07.933186 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:07.933116    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba2f1b97-4d8d-4b69-bf55-531600544660-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:07.933353 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:07.933200    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba2f1b97-4d8d-4b69-bf55-531600544660-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:07.933353 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:07.933250    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba2f1b97-4d8d-4b69-bf55-531600544660-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:07.933353 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:07.933299    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba2f1b97-4d8d-4b69-bf55-531600544660-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:07.933353 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:07.933338    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba2f1b97-4d8d-4b69-bf55-531600544660-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:07.933573 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:07.933389    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9w2b\" (UniqueName: \"kubernetes.io/projected/ba2f1b97-4d8d-4b69-bf55-531600544660-kube-api-access-f9w2b\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:07.938273 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:07.938244    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"]
Nov 04 03:40:08.034468 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.034426    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba2f1b97-4d8d-4b69-bf55-531600544660-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.034667 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.034516    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba2f1b97-4d8d-4b69-bf55-531600544660-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.034667 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.034558    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba2f1b97-4d8d-4b69-bf55-531600544660-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.034667 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.034604    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba2f1b97-4d8d-4b69-bf55-531600544660-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.034667 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.034640    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba2f1b97-4d8d-4b69-bf55-531600544660-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.034911 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.034685    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9w2b\" (UniqueName: \"kubernetes.io/projected/ba2f1b97-4d8d-4b69-bf55-531600544660-kube-api-access-f9w2b\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.034911 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.034771    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba2f1b97-4d8d-4b69-bf55-531600544660-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.034911 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.034815    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba2f1b97-4d8d-4b69-bf55-531600544660-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.034911 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.034904    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba2f1b97-4d8d-4b69-bf55-531600544660-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.035111 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.034899    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba2f1b97-4d8d-4b69-bf55-531600544660-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.035218 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.035195    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba2f1b97-4d8d-4b69-bf55-531600544660-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.035279 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.035254    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba2f1b97-4d8d-4b69-bf55-531600544660-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.035334 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.035286    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba2f1b97-4d8d-4b69-bf55-531600544660-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.035689 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.035659    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba2f1b97-4d8d-4b69-bf55-531600544660-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.037038 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.037013    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba2f1b97-4d8d-4b69-bf55-531600544660-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.037262 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.037244    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba2f1b97-4d8d-4b69-bf55-531600544660-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.097714 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.097643    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba2f1b97-4d8d-4b69-bf55-531600544660-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.097898 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.097834    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9w2b\" (UniqueName: \"kubernetes.io/projected/ba2f1b97-4d8d-4b69-bf55-531600544660-kube-api-access-f9w2b\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-dh692\" (UID: \"ba2f1b97-4d8d-4b69-bf55-531600544660\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.145653 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.145623    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:08.580116 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.580087    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"]
Nov 04 03:40:08.581099 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:40:08.581068    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba2f1b97_4d8d_4b69_bf55_531600544660.slice/crio-158c748cfe566fd290d17bfbb4a55b8a3cde609c3f5ebf6705d8ed1e1b11ab4b WatchSource:0}: Error finding container 158c748cfe566fd290d17bfbb4a55b8a3cde609c3f5ebf6705d8ed1e1b11ab4b: Status 404 returned error can't find the container with id 158c748cfe566fd290d17bfbb4a55b8a3cde609c3f5ebf6705d8ed1e1b11ab4b
Nov 04 03:40:08.583516 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.583480    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:40:08.583620 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.583560    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:40:08.583620 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.583607    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:40:08.818037 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.817995    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692" event={"ID":"ba2f1b97-4d8d-4b69-bf55-531600544660","Type":"ContainerStarted","Data":"03e512d18d193fd390b3fac551bb854f9989a400408dbf8a62c41deb446f295a"}
Nov 04 03:40:08.818196 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.818062    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692" event={"ID":"ba2f1b97-4d8d-4b69-bf55-531600544660","Type":"ContainerStarted","Data":"158c748cfe566fd290d17bfbb4a55b8a3cde609c3f5ebf6705d8ed1e1b11ab4b"}
Nov 04 03:40:08.825051 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.824009    4521 generic.go:356] "Generic (PLEG): container finished" podID="1b7602b8-2d7b-42b4-acd3-868233206892" containerID="6c97d4c14e24737ac15ca296dcd21fc061e0ec1c653e4ef8f99048d8436655a3" exitCode=0
Nov 04 03:40:08.825051 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.824160    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq" event={"ID":"1b7602b8-2d7b-42b4-acd3-868233206892","Type":"ContainerDied","Data":"6c97d4c14e24737ac15ca296dcd21fc061e0ec1c653e4ef8f99048d8436655a3"}
Nov 04 03:40:08.890131 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.890097    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"
Nov 04 03:40:08.944044 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.943984    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692" podStartSLOduration=1.943967283 podStartE2EDuration="1.943967283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:40:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:40:08.940952401 +0000 UTC m=+1375.145552724" watchObservedRunningTime="2026-11-04 03:40:08.943967283 +0000 UTC m=+1375.148567586"
Nov 04 03:40:08.944785 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.944758    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b7602b8-2d7b-42b4-acd3-868233206892-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7602b8-2d7b-42b4-acd3-868233206892-plugins-config-volume\") pod \"1b7602b8-2d7b-42b4-acd3-868233206892\" (UID: \"1b7602b8-2d7b-42b4-acd3-868233206892\") "
Nov 04 03:40:08.944968 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.944941    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b7602b8-2d7b-42b4-acd3-868233206892-kube-api-access-cdjqm\" (UniqueName: \"kubernetes.io/projected/1b7602b8-2d7b-42b4-acd3-868233206892-kube-api-access-cdjqm\") pod \"1b7602b8-2d7b-42b4-acd3-868233206892\" (UID: \"1b7602b8-2d7b-42b4-acd3-868233206892\") "
Nov 04 03:40:08.945247 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.945224    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b7602b8-2d7b-42b4-acd3-868233206892-plugins-config-volume" pod "1b7602b8-2d7b-42b4-acd3-868233206892" (UID: "1b7602b8-2d7b-42b4-acd3-868233206892"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:40:08.945353 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.945341    4521 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7602b8-2d7b-42b4-acd3-868233206892-plugins-config-volume\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:08.947002 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:08.946982    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b7602b8-2d7b-42b4-acd3-868233206892-kube-api-access-cdjqm" pod "1b7602b8-2d7b-42b4-acd3-868233206892" (UID: "1b7602b8-2d7b-42b4-acd3-868233206892"). InnerVolumeSpecName "kube-api-access-cdjqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:09.046860 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:09.046806    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdjqm\" (UniqueName: \"kubernetes.io/projected/1b7602b8-2d7b-42b4-acd3-868233206892-kube-api-access-cdjqm\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:09.146060 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:09.145982    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:09.147490 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:09.147462    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-dh692 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 03:40:09.147628 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:09.147519    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692" podUID="ba2f1b97-4d8d-4b69-bf55-531600544660" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 03:40:09.832081 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:09.832042    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq" event={"ID":"1b7602b8-2d7b-42b4-acd3-868233206892","Type":"ContainerDied","Data":"76afaf7d94e7f37e3817d1a9f7b41bd76b0a139f96ffa6b4155b5635b32e518a"}
Nov 04 03:40:09.832323 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:09.832060    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"
Nov 04 03:40:09.832323 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:09.832150    4521 scope.go:122] "RemoveContainer" containerID="6c97d4c14e24737ac15ca296dcd21fc061e0ec1c653e4ef8f99048d8436655a3"
Nov 04 03:40:10.023214 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:10.023180    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"]
Nov 04 03:40:10.072808 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:10.072773    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-ee1947-bdbb8c7bc-jvjjq"]
Nov 04 03:40:10.146903 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:10.146863    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-dh692 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 03:40:10.147105 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:10.146923    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692" podUID="ba2f1b97-4d8d-4b69-bf55-531600544660" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 03:40:11.146293 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:11.146254    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-dh692 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 03:40:11.146790 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:11.146308    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692" podUID="ba2f1b97-4d8d-4b69-bf55-531600544660" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 03:40:11.916275 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:11.916241    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b7602b8-2d7b-42b4-acd3-868233206892" path="/var/lib/kubelet/pods/1b7602b8-2d7b-42b4-acd3-868233206892/volumes"
Nov 04 03:40:12.146285 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:12.146248    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-dh692 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 03:40:12.146467 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:12.146310    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692" podUID="ba2f1b97-4d8d-4b69-bf55-531600544660" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 03:40:13.146340 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:13.146307    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-dh692 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 03:40:13.146524 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:13.146363    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692" podUID="ba2f1b97-4d8d-4b69-bf55-531600544660" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 03:40:14.146202 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:14.146174    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-dh692 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 03:40:14.146301 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:14.146236    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692" podUID="ba2f1b97-4d8d-4b69-bf55-531600544660" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 03:40:14.771171 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:14.771136    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:40:15.146219 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:15.146135    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-dh692 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 03:40:15.146374 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:15.146219    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692" podUID="ba2f1b97-4d8d-4b69-bf55-531600544660" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 03:40:16.147041 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:16.146995    4521 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-dh692 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 03:40:16.147441 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:16.147067    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692" podUID="ba2f1b97-4d8d-4b69-bf55-531600544660" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 03:40:17.127994 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.127955    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"]
Nov 04 03:40:17.128197 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.128164    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt" podUID="3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7" containerName="maas-api" containerID="cri-o://afe5190f0a335a1d6efcffd5e1a01bd03818dddb50880ff9e3247339bb8dea0a" gracePeriod=30
Nov 04 03:40:17.432484 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.432456    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:40:17.519695 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.519659    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-maas-api-tls\") pod \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\" (UID: \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\") "
Nov 04 03:40:17.519835 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.519749    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-metrics-certs\") pod \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\" (UID: \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\") "
Nov 04 03:40:17.519967 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.519954    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-kube-api-access-jcd8r\" (UniqueName: \"kubernetes.io/projected/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-kube-api-access-jcd8r\") pod \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\" (UID: \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\") "
Nov 04 03:40:17.520011 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.520000    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-openshift-service-ca\") pod \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\" (UID: \"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7\") "
Nov 04 03:40:17.520448 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.520412    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-openshift-service-ca" pod "3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7" (UID: "3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:40:17.521936 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.521910    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-maas-api-tls" pod "3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7" (UID: "3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:40:17.522066 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.522022    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-metrics-certs" pod "3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7" (UID: "3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:40:17.522171 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.522146    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-kube-api-access-jcd8r" pod "3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7" (UID: "3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7"). InnerVolumeSpecName "kube-api-access-jcd8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:17.544862 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.542085    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:17.544862 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.543559    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:17.556210 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.556144    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-dh692"
Nov 04 03:40:17.621682 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.621649    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcd8r\" (UniqueName: \"kubernetes.io/projected/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-kube-api-access-jcd8r\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:17.621682 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.621677    4521 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-openshift-service-ca\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:17.621682 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.621687    4521 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-maas-api-tls\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:17.622031 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.621696    4521 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7-metrics-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:17.884103 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.884071    4521 generic.go:356] "Generic (PLEG): container finished" podID="3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7" containerID="afe5190f0a335a1d6efcffd5e1a01bd03818dddb50880ff9e3247339bb8dea0a" exitCode=0
Nov 04 03:40:17.884301 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.884158    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"
Nov 04 03:40:17.884301 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.884186    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt" event={"ID":"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7","Type":"ContainerDied","Data":"afe5190f0a335a1d6efcffd5e1a01bd03818dddb50880ff9e3247339bb8dea0a"}
Nov 04 03:40:17.884301 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.884248    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt" event={"ID":"3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7","Type":"ContainerDied","Data":"d037ea2a29df03a7b7f99e58afb045bf690428d8e4be49316d188242e60fd504"}
Nov 04 03:40:17.884301 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.884282    4521 scope.go:122] "RemoveContainer" containerID="afe5190f0a335a1d6efcffd5e1a01bd03818dddb50880ff9e3247339bb8dea0a"
Nov 04 03:40:17.895782 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.895750    4521 scope.go:122] "RemoveContainer" containerID="afe5190f0a335a1d6efcffd5e1a01bd03818dddb50880ff9e3247339bb8dea0a"
Nov 04 03:40:17.896052 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:40:17.896031    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afe5190f0a335a1d6efcffd5e1a01bd03818dddb50880ff9e3247339bb8dea0a\": container with ID starting with afe5190f0a335a1d6efcffd5e1a01bd03818dddb50880ff9e3247339bb8dea0a not found: ID does not exist" containerID="afe5190f0a335a1d6efcffd5e1a01bd03818dddb50880ff9e3247339bb8dea0a"
Nov 04 03:40:17.896136 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:17.896060    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afe5190f0a335a1d6efcffd5e1a01bd03818dddb50880ff9e3247339bb8dea0a"} err="failed to get container status \"afe5190f0a335a1d6efcffd5e1a01bd03818dddb50880ff9e3247339bb8dea0a\": rpc error: code = NotFound desc = could not find container \"afe5190f0a335a1d6efcffd5e1a01bd03818dddb50880ff9e3247339bb8dea0a\": container with ID starting with afe5190f0a335a1d6efcffd5e1a01bd03818dddb50880ff9e3247339bb8dea0a not found: ID does not exist"
Nov 04 03:40:18.052471 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:18.052437    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"]
Nov 04 03:40:18.056061 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:18.056033    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-ee1947-dcf79cf8f-9qhwt"]
Nov 04 03:40:19.912499 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:19.912463    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7" path="/var/lib/kubelet/pods/3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7/volumes"
Nov 04 03:40:20.997050 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:20.997016    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"]
Nov 04 03:40:20.997503 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:20.997214    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc" podUID="30200904-1724-4ddd-aed5-2e9b612c26e9" containerName="istio-proxy" containerID="cri-o://66040061d78c078a6eef436293266a640d1d3d69d63df2ef397c445b10fba221" gracePeriod=30
Nov 04 03:40:21.017398 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:21.017367    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"]
Nov 04 03:40:21.017600 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:21.017576    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd" podUID="934b3519-cc81-4ea9-b8bd-9e962e9fcacb" containerName="istio-proxy" containerID="cri-o://6db1f1734c6cfcd5d2162b8596c356d76f555a121afebafe31a59452cd80b1a7" gracePeriod=30
Nov 04 03:40:21.047132 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:21.047097    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"]
Nov 04 03:40:21.047321 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:21.047291    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm" podUID="efc6e068-c4fc-4559-a531-d62af03cbc9b" containerName="istio-proxy" containerID="cri-o://a9c7da8a88d1fba93b2262df8ce38e93dc8c50b66a100d614734f8027249ed11" gracePeriod=30
Nov 04 03:40:21.822462 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:21.822423    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:40:21.822656 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:21.822479    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm" podUID="efc6e068-c4fc-4559-a531-d62af03cbc9b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:40:22.545940 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:22.545906    4521 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:40:22.546338 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:22.545964    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd" podUID="934b3519-cc81-4ea9-b8bd-9e962e9fcacb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:40:24.249749 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.249713    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"]
Nov 04 03:40:24.250339 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.250324    4521 cpu_manager.go:414] "containerMap: removing container" podUID="1b7602b8-2d7b-42b4-acd3-868233206892" containerName="payload-pre-processing"
Nov 04 03:40:24.250390 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.250340    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b7602b8-2d7b-42b4-acd3-868233206892" containerName="payload-pre-processing"
Nov 04 03:40:24.250390 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.250354    4521 cpu_manager.go:414] "containerMap: removing container" podUID="3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7" containerName="maas-api"
Nov 04 03:40:24.250390 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.250365    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7" containerName="maas-api"
Nov 04 03:40:24.250482 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.250470    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b7602b8-2d7b-42b4-acd3-868233206892" containerName="payload-pre-processing"
Nov 04 03:40:24.250482 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.250480    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c67035d-6e80-4b8b-9c0e-5dc3a1b4b4a7" containerName="maas-api"
Nov 04 03:40:24.255352 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.255336    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:40:24.304940 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.304905    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53e6e076-af21-469c-9713-6b7f608833bc-maas-api-tls\") pod \"maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s\" (UID: \"53e6e076-af21-469c-9713-6b7f608833bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:40:24.305075 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.304943    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53e6e076-af21-469c-9713-6b7f608833bc-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s\" (UID: \"53e6e076-af21-469c-9713-6b7f608833bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:40:24.305075 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.304979    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53e6e076-af21-469c-9713-6b7f608833bc-metrics-certs\") pod \"maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s\" (UID: \"53e6e076-af21-469c-9713-6b7f608833bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:40:24.305227 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.305204    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt8sm\" (UniqueName: \"kubernetes.io/projected/53e6e076-af21-469c-9713-6b7f608833bc-kube-api-access-lt8sm\") pod \"maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s\" (UID: \"53e6e076-af21-469c-9713-6b7f608833bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:40:24.309211 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.309191    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-951893-serving-cert\""
Nov 04 03:40:24.363054 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.363024    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"]
Nov 04 03:40:24.406550 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.406512    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt8sm\" (UniqueName: \"kubernetes.io/projected/53e6e076-af21-469c-9713-6b7f608833bc-kube-api-access-lt8sm\") pod \"maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s\" (UID: \"53e6e076-af21-469c-9713-6b7f608833bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:40:24.406723 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.406598    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53e6e076-af21-469c-9713-6b7f608833bc-maas-api-tls\") pod \"maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s\" (UID: \"53e6e076-af21-469c-9713-6b7f608833bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:40:24.406723 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:40:24.406699    4521 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-serving-cert: secret "maas-api-e2e-extmod-b-951893-serving-cert" not found
Nov 04 03:40:24.406855 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:40:24.406756    4521 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53e6e076-af21-469c-9713-6b7f608833bc-maas-api-tls podName:53e6e076-af21-469c-9713-6b7f608833bc nodeName:}" failed. No retries permitted until 2026-11-04 03:40:24.906737589 +0000 UTC m=+1391.111337874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/53e6e076-af21-469c-9713-6b7f608833bc-maas-api-tls") pod "maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s" (UID: "53e6e076-af21-469c-9713-6b7f608833bc") : secret "maas-api-e2e-extmod-b-951893-serving-cert" not found
Nov 04 03:40:24.406855 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.406757    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53e6e076-af21-469c-9713-6b7f608833bc-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s\" (UID: \"53e6e076-af21-469c-9713-6b7f608833bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:40:24.406855 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.406812    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53e6e076-af21-469c-9713-6b7f608833bc-metrics-certs\") pod \"maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s\" (UID: \"53e6e076-af21-469c-9713-6b7f608833bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:40:24.407590 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.407564    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53e6e076-af21-469c-9713-6b7f608833bc-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s\" (UID: \"53e6e076-af21-469c-9713-6b7f608833bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:40:24.409607 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.409579    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53e6e076-af21-469c-9713-6b7f608833bc-metrics-certs\") pod \"maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s\" (UID: \"53e6e076-af21-469c-9713-6b7f608833bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:40:24.468081 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.468057    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt8sm\" (UniqueName: \"kubernetes.io/projected/53e6e076-af21-469c-9713-6b7f608833bc-kube-api-access-lt8sm\") pod \"maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s\" (UID: \"53e6e076-af21-469c-9713-6b7f608833bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:40:24.912719 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.912678    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53e6e076-af21-469c-9713-6b7f608833bc-maas-api-tls\") pod \"maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s\" (UID: \"53e6e076-af21-469c-9713-6b7f608833bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:40:24.915058 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:24.915032    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53e6e076-af21-469c-9713-6b7f608833bc-maas-api-tls\") pod \"maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s\" (UID: \"53e6e076-af21-469c-9713-6b7f608833bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:40:25.168699 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:25.168618    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:40:25.545528 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:25.545498    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"]
Nov 04 03:40:25.546925 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:40:25.546899    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53e6e076_af21_469c_9713_6b7f608833bc.slice/crio-fad7dffb8eda685843ae5157bb19507c1e00af69f9a71b543c3728897be79d58 WatchSource:0}: Error finding container fad7dffb8eda685843ae5157bb19507c1e00af69f9a71b543c3728897be79d58: Status 404 returned error can't find the container with id fad7dffb8eda685843ae5157bb19507c1e00af69f9a71b543c3728897be79d58
Nov 04 03:40:25.934042 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:25.933931    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s" event={"ID":"53e6e076-af21-469c-9713-6b7f608833bc","Type":"ContainerStarted","Data":"fad7dffb8eda685843ae5157bb19507c1e00af69f9a71b543c3728897be79d58"}
Nov 04 03:40:26.323503 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.323468    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:40:26.338529 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.338501    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:40:26.344946 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.344927    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:40:26.434330 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.434296    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istiod-ca-cert\") pod \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") "
Nov 04 03:40:26.434533 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.434340    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-data\") pod \"30200904-1724-4ddd-aed5-2e9b612c26e9\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") "
Nov 04 03:40:26.434533 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.434367    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30200904-1724-4ddd-aed5-2e9b612c26e9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30200904-1724-4ddd-aed5-2e9b612c26e9-istiod-ca-cert\") pod \"30200904-1724-4ddd-aed5-2e9b612c26e9\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") "
Nov 04 03:40:26.434533 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.434391    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-envoy\") pod \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") "
Nov 04 03:40:26.434533 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.434453    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-podinfo\") pod \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") "
Nov 04 03:40:26.434533 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.434509    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-envoy\") pod \"efc6e068-c4fc-4559-a531-d62af03cbc9b\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") "
Nov 04 03:40:26.434806 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.434547    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-workload-socket\") pod \"efc6e068-c4fc-4559-a531-d62af03cbc9b\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") "
Nov 04 03:40:26.434806 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.434662    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-data" pod "30200904-1724-4ddd-aed5-2e9b612c26e9" (UID: "30200904-1724-4ddd-aed5-2e9b612c26e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.434806 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.434722    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30200904-1724-4ddd-aed5-2e9b612c26e9-kube-api-access-hpsvv\" (UniqueName: \"kubernetes.io/projected/30200904-1724-4ddd-aed5-2e9b612c26e9-kube-api-access-hpsvv\") pod \"30200904-1724-4ddd-aed5-2e9b612c26e9\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") "
Nov 04 03:40:26.434977 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.434830    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efc6e068-c4fc-4559-a531-d62af03cbc9b-kube-api-access-c25gn\" (UniqueName: \"kubernetes.io/projected/efc6e068-c4fc-4559-a531-d62af03cbc9b-kube-api-access-c25gn\") pod \"efc6e068-c4fc-4559-a531-d62af03cbc9b\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") "
Nov 04 03:40:26.434977 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.434896    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-workload-socket" pod "efc6e068-c4fc-4559-a531-d62af03cbc9b" (UID: "efc6e068-c4fc-4559-a531-d62af03cbc9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.434977 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.434917    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30200904-1724-4ddd-aed5-2e9b612c26e9-istiod-ca-cert" pod "30200904-1724-4ddd-aed5-2e9b612c26e9" (UID: "30200904-1724-4ddd-aed5-2e9b612c26e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:40:26.434977 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.434917    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istiod-ca-cert" pod "934b3519-cc81-4ea9-b8bd-9e962e9fcacb" (UID: "934b3519-cc81-4ea9-b8bd-9e962e9fcacb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:40:26.434977 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.434947    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-data\") pod \"efc6e068-c4fc-4559-a531-d62af03cbc9b\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") "
Nov 04 03:40:26.435218 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.434993    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-workload-certs\") pod \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") "
Nov 04 03:40:26.435218 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435025    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-workload-socket\") pod \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") "
Nov 04 03:40:26.435218 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435074    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-workload-socket\") pod \"30200904-1724-4ddd-aed5-2e9b612c26e9\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") "
Nov 04 03:40:26.435218 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435139    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-data\") pod \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") "
Nov 04 03:40:26.435218 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435183    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-workload-certs\") pod \"30200904-1724-4ddd-aed5-2e9b612c26e9\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") "
Nov 04 03:40:26.435218 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435199    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-data" pod "efc6e068-c4fc-4559-a531-d62af03cbc9b" (UID: "efc6e068-c4fc-4559-a531-d62af03cbc9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.435517 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435223    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-token\" (UniqueName: \"kubernetes.io/projected/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-token\") pod \"efc6e068-c4fc-4559-a531-d62af03cbc9b\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") "
Nov 04 03:40:26.435517 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435270    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-token\" (UniqueName: \"kubernetes.io/projected/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-token\") pod \"30200904-1724-4ddd-aed5-2e9b612c26e9\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") "
Nov 04 03:40:26.435517 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435296    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-credential-socket\") pod \"30200904-1724-4ddd-aed5-2e9b612c26e9\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") "
Nov 04 03:40:26.435517 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435327    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-workload-certs\") pod \"efc6e068-c4fc-4559-a531-d62af03cbc9b\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") "
Nov 04 03:40:26.435517 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435359    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-credential-socket\") pod \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") "
Nov 04 03:40:26.435517 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435362    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-workload-socket" pod "934b3519-cc81-4ea9-b8bd-9e962e9fcacb" (UID: "934b3519-cc81-4ea9-b8bd-9e962e9fcacb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.435517 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435405    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-envoy\") pod \"30200904-1724-4ddd-aed5-2e9b612c26e9\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") "
Nov 04 03:40:26.435517 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435447    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-data" pod "934b3519-cc81-4ea9-b8bd-9e962e9fcacb" (UID: "934b3519-cc81-4ea9-b8bd-9e962e9fcacb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.435517 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435473    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-kube-api-access-zjcbp\" (UniqueName: \"kubernetes.io/projected/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-kube-api-access-zjcbp\") pod \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") "
Nov 04 03:40:26.435975 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435523    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-token\" (UniqueName: \"kubernetes.io/projected/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-token\") pod \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\" (UID: \"934b3519-cc81-4ea9-b8bd-9e962e9fcacb\") "
Nov 04 03:40:26.435975 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435524    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-workload-socket" pod "30200904-1724-4ddd-aed5-2e9b612c26e9" (UID: "30200904-1724-4ddd-aed5-2e9b612c26e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.435975 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435567    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-podinfo\") pod \"30200904-1724-4ddd-aed5-2e9b612c26e9\" (UID: \"30200904-1724-4ddd-aed5-2e9b612c26e9\") "
Nov 04 03:40:26.435975 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435596    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-podinfo\") pod \"efc6e068-c4fc-4559-a531-d62af03cbc9b\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") "
Nov 04 03:40:26.435975 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435634    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/efc6e068-c4fc-4559-a531-d62af03cbc9b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efc6e068-c4fc-4559-a531-d62af03cbc9b-istiod-ca-cert\") pod \"efc6e068-c4fc-4559-a531-d62af03cbc9b\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") "
Nov 04 03:40:26.435975 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435670    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-credential-socket\") pod \"efc6e068-c4fc-4559-a531-d62af03cbc9b\" (UID: \"efc6e068-c4fc-4559-a531-d62af03cbc9b\") "
Nov 04 03:40:26.435975 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.435886    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-workload-certs" pod "efc6e068-c4fc-4559-a531-d62af03cbc9b" (UID: "efc6e068-c4fc-4559-a531-d62af03cbc9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.436359 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.436091    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-credential-socket" pod "30200904-1724-4ddd-aed5-2e9b612c26e9" (UID: "30200904-1724-4ddd-aed5-2e9b612c26e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.436359 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.436118    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-workload-certs" pod "30200904-1724-4ddd-aed5-2e9b612c26e9" (UID: "30200904-1724-4ddd-aed5-2e9b612c26e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.436658    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-workload-certs" pod "934b3519-cc81-4ea9-b8bd-9e962e9fcacb" (UID: "934b3519-cc81-4ea9-b8bd-9e962e9fcacb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.436934    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-credential-socket" pod "934b3519-cc81-4ea9-b8bd-9e962e9fcacb" (UID: "934b3519-cc81-4ea9-b8bd-9e962e9fcacb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437002    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efc6e068-c4fc-4559-a531-d62af03cbc9b-istiod-ca-cert" pod "efc6e068-c4fc-4559-a531-d62af03cbc9b" (UID: "efc6e068-c4fc-4559-a531-d62af03cbc9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437084    4521 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-data\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437103    4521 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-workload-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437119    4521 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-credential-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437132    4521 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-workload-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437141    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-credential-socket" pod "efc6e068-c4fc-4559-a531-d62af03cbc9b" (UID: "efc6e068-c4fc-4559-a531-d62af03cbc9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437147    4521 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-credential-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437183    4521 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istiod-ca-cert\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437200    4521 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-data\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437199    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-envoy" pod "934b3519-cc81-4ea9-b8bd-9e962e9fcacb" (UID: "934b3519-cc81-4ea9-b8bd-9e962e9fcacb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437215    4521 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30200904-1724-4ddd-aed5-2e9b612c26e9-istiod-ca-cert\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437231    4521 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-workload-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437246    4521 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-data\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437263    4521 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-workload-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437280    4521 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-workload-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.437319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437294    4521 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-workload-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.438352 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437663    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-podinfo" pod "934b3519-cc81-4ea9-b8bd-9e962e9fcacb" (UID: "934b3519-cc81-4ea9-b8bd-9e962e9fcacb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:40:26.438352 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.437805    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efc6e068-c4fc-4559-a531-d62af03cbc9b-kube-api-access-c25gn" pod "efc6e068-c4fc-4559-a531-d62af03cbc9b" (UID: "efc6e068-c4fc-4559-a531-d62af03cbc9b"). InnerVolumeSpecName "kube-api-access-c25gn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.438986 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.438953    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-envoy" pod "efc6e068-c4fc-4559-a531-d62af03cbc9b" (UID: "efc6e068-c4fc-4559-a531-d62af03cbc9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.439166 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.439142    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-podinfo" pod "efc6e068-c4fc-4559-a531-d62af03cbc9b" (UID: "efc6e068-c4fc-4559-a531-d62af03cbc9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:40:26.439301 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.439201    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-envoy" pod "30200904-1724-4ddd-aed5-2e9b612c26e9" (UID: "30200904-1724-4ddd-aed5-2e9b612c26e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.439301 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.439235    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-token" pod "30200904-1724-4ddd-aed5-2e9b612c26e9" (UID: "30200904-1724-4ddd-aed5-2e9b612c26e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.439415 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.439365    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-kube-api-access-zjcbp" pod "934b3519-cc81-4ea9-b8bd-9e962e9fcacb" (UID: "934b3519-cc81-4ea9-b8bd-9e962e9fcacb"). InnerVolumeSpecName "kube-api-access-zjcbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.439469 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.439427    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30200904-1724-4ddd-aed5-2e9b612c26e9-kube-api-access-hpsvv" pod "30200904-1724-4ddd-aed5-2e9b612c26e9" (UID: "30200904-1724-4ddd-aed5-2e9b612c26e9"). InnerVolumeSpecName "kube-api-access-hpsvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.439599 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.439581    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-podinfo" pod "30200904-1724-4ddd-aed5-2e9b612c26e9" (UID: "30200904-1724-4ddd-aed5-2e9b612c26e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:40:26.440141 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.440127    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-token" pod "efc6e068-c4fc-4559-a531-d62af03cbc9b" (UID: "efc6e068-c4fc-4559-a531-d62af03cbc9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.440662 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.440636    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-token" pod "934b3519-cc81-4ea9-b8bd-9e962e9fcacb" (UID: "934b3519-cc81-4ea9-b8bd-9e962e9fcacb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.538751 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.538715    4521 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-token\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.538983 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.538754    4521 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-envoy\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.538983 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.538770    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjcbp\" (UniqueName: \"kubernetes.io/projected/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-kube-api-access-zjcbp\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.538983 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.538785    4521 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-token\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.538983 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.538801    4521 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30200904-1724-4ddd-aed5-2e9b612c26e9-istio-podinfo\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.538983 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.538815    4521 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-podinfo\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.538983 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.538829    4521 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efc6e068-c4fc-4559-a531-d62af03cbc9b-istiod-ca-cert\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.538983 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.538860    4521 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-credential-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.538983 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.538875    4521 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-envoy\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.538983 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.538889    4521 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934b3519-cc81-4ea9-b8bd-9e962e9fcacb-istio-podinfo\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.538983 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.538904    4521 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-envoy\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.538983 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.538919    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpsvv\" (UniqueName: \"kubernetes.io/projected/30200904-1724-4ddd-aed5-2e9b612c26e9-kube-api-access-hpsvv\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.538983 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.538934    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c25gn\" (UniqueName: \"kubernetes.io/projected/efc6e068-c4fc-4559-a531-d62af03cbc9b-kube-api-access-c25gn\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.538983 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.538948    4521 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efc6e068-c4fc-4559-a531-d62af03cbc9b-istio-token\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.941096 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.941064    4521 generic.go:356] "Generic (PLEG): container finished" podID="30200904-1724-4ddd-aed5-2e9b612c26e9" containerID="66040061d78c078a6eef436293266a640d1d3d69d63df2ef397c445b10fba221" exitCode=0
Nov 04 03:40:26.941648 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.941127    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"
Nov 04 03:40:26.941648 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.941144    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc" event={"ID":"30200904-1724-4ddd-aed5-2e9b612c26e9","Type":"ContainerDied","Data":"66040061d78c078a6eef436293266a640d1d3d69d63df2ef397c445b10fba221"}
Nov 04 03:40:26.941648 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.941204    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc" event={"ID":"30200904-1724-4ddd-aed5-2e9b612c26e9","Type":"ContainerDied","Data":"4290e0cbb71b9cb85c5f5c408fd35b76c0dd39b4ca4a5b335e685ec842553f74"}
Nov 04 03:40:26.941648 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.941236    4521 scope.go:122] "RemoveContainer" containerID="66040061d78c078a6eef436293266a640d1d3d69d63df2ef397c445b10fba221"
Nov 04 03:40:26.942583 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.942557    4521 generic.go:356] "Generic (PLEG): container finished" podID="934b3519-cc81-4ea9-b8bd-9e962e9fcacb" containerID="6db1f1734c6cfcd5d2162b8596c356d76f555a121afebafe31a59452cd80b1a7" exitCode=0
Nov 04 03:40:26.942770 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.942588    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd" event={"ID":"934b3519-cc81-4ea9-b8bd-9e962e9fcacb","Type":"ContainerDied","Data":"6db1f1734c6cfcd5d2162b8596c356d76f555a121afebafe31a59452cd80b1a7"}
Nov 04 03:40:26.942985 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.942806    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd" event={"ID":"934b3519-cc81-4ea9-b8bd-9e962e9fcacb","Type":"ContainerDied","Data":"f87c82347c1e7fb4eb9d95f7c26d9701d50347d60323cddedff5f5f372d7a8f3"}
Nov 04 03:40:26.942985 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.942623    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"
Nov 04 03:40:26.944825 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.944797    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s" event={"ID":"53e6e076-af21-469c-9713-6b7f608833bc","Type":"ContainerStarted","Data":"8065a90ffcd9e50a11628dad6fb2f728872733d1053ce9f46064d69fea701830"}
Nov 04 03:40:26.944936 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.944874    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:40:26.946459 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.946439    4521 generic.go:356] "Generic (PLEG): container finished" podID="efc6e068-c4fc-4559-a531-d62af03cbc9b" containerID="a9c7da8a88d1fba93b2262df8ce38e93dc8c50b66a100d614734f8027249ed11" exitCode=0
Nov 04 03:40:26.946571 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.946494    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"
Nov 04 03:40:26.946571 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.946517    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm" event={"ID":"efc6e068-c4fc-4559-a531-d62af03cbc9b","Type":"ContainerDied","Data":"a9c7da8a88d1fba93b2262df8ce38e93dc8c50b66a100d614734f8027249ed11"}
Nov 04 03:40:26.946571 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.946551    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm" event={"ID":"efc6e068-c4fc-4559-a531-d62af03cbc9b","Type":"ContainerDied","Data":"1641eb3d73d0c5c7d91069cf20e5135911b9fc11a74414aab3232b2dcb0dbf7f"}
Nov 04 03:40:26.953079 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.953061    4521 scope.go:122] "RemoveContainer" containerID="66040061d78c078a6eef436293266a640d1d3d69d63df2ef397c445b10fba221"
Nov 04 03:40:26.953318 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:40:26.953300    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66040061d78c078a6eef436293266a640d1d3d69d63df2ef397c445b10fba221\": container with ID starting with 66040061d78c078a6eef436293266a640d1d3d69d63df2ef397c445b10fba221 not found: ID does not exist" containerID="66040061d78c078a6eef436293266a640d1d3d69d63df2ef397c445b10fba221"
Nov 04 03:40:26.953387 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.953324    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66040061d78c078a6eef436293266a640d1d3d69d63df2ef397c445b10fba221"} err="failed to get container status \"66040061d78c078a6eef436293266a640d1d3d69d63df2ef397c445b10fba221\": rpc error: code = NotFound desc = could not find container \"66040061d78c078a6eef436293266a640d1d3d69d63df2ef397c445b10fba221\": container with ID starting with 66040061d78c078a6eef436293266a640d1d3d69d63df2ef397c445b10fba221 not found: ID does not exist"
Nov 04 03:40:26.953387 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.953340    4521 scope.go:122] "RemoveContainer" containerID="6db1f1734c6cfcd5d2162b8596c356d76f555a121afebafe31a59452cd80b1a7"
Nov 04 03:40:26.962791 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.962776    4521 scope.go:122] "RemoveContainer" containerID="6db1f1734c6cfcd5d2162b8596c356d76f555a121afebafe31a59452cd80b1a7"
Nov 04 03:40:26.963053 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:40:26.963035    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6db1f1734c6cfcd5d2162b8596c356d76f555a121afebafe31a59452cd80b1a7\": container with ID starting with 6db1f1734c6cfcd5d2162b8596c356d76f555a121afebafe31a59452cd80b1a7 not found: ID does not exist" containerID="6db1f1734c6cfcd5d2162b8596c356d76f555a121afebafe31a59452cd80b1a7"
Nov 04 03:40:26.963105 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.963060    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6db1f1734c6cfcd5d2162b8596c356d76f555a121afebafe31a59452cd80b1a7"} err="failed to get container status \"6db1f1734c6cfcd5d2162b8596c356d76f555a121afebafe31a59452cd80b1a7\": rpc error: code = NotFound desc = could not find container \"6db1f1734c6cfcd5d2162b8596c356d76f555a121afebafe31a59452cd80b1a7\": container with ID starting with 6db1f1734c6cfcd5d2162b8596c356d76f555a121afebafe31a59452cd80b1a7 not found: ID does not exist"
Nov 04 03:40:26.963105 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.963077    4521 scope.go:122] "RemoveContainer" containerID="a9c7da8a88d1fba93b2262df8ce38e93dc8c50b66a100d614734f8027249ed11"
Nov 04 03:40:26.971817 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.971798    4521 scope.go:122] "RemoveContainer" containerID="a9c7da8a88d1fba93b2262df8ce38e93dc8c50b66a100d614734f8027249ed11"
Nov 04 03:40:26.972095 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:40:26.972077    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9c7da8a88d1fba93b2262df8ce38e93dc8c50b66a100d614734f8027249ed11\": container with ID starting with a9c7da8a88d1fba93b2262df8ce38e93dc8c50b66a100d614734f8027249ed11 not found: ID does not exist" containerID="a9c7da8a88d1fba93b2262df8ce38e93dc8c50b66a100d614734f8027249ed11"
Nov 04 03:40:26.972147 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:26.972103    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9c7da8a88d1fba93b2262df8ce38e93dc8c50b66a100d614734f8027249ed11"} err="failed to get container status \"a9c7da8a88d1fba93b2262df8ce38e93dc8c50b66a100d614734f8027249ed11\": rpc error: code = NotFound desc = could not find container \"a9c7da8a88d1fba93b2262df8ce38e93dc8c50b66a100d614734f8027249ed11\": container with ID starting with a9c7da8a88d1fba93b2262df8ce38e93dc8c50b66a100d614734f8027249ed11 not found: ID does not exist"
Nov 04 03:40:27.110310 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:27.110276    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"]
Nov 04 03:40:27.114085 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:27.114055    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-2t9nc"]
Nov 04 03:40:27.226151 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:27.226065    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"]
Nov 04 03:40:27.230831 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:27.230798    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-dvcgm"]
Nov 04 03:40:27.352013 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:27.351968    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"]
Nov 04 03:40:27.354337 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:27.354307    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-6rqkd"]
Nov 04 03:40:27.464427 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:27.464371    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s" podStartSLOduration=2.522888305 podStartE2EDuration="3.464356416s" totalImagesPullingTime="941.468111ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:40:27.461864277 +0000 UTC m=+1393.666464572" watchObservedRunningTime="2026-11-04 03:40:27.464356416 +0000 UTC m=+1393.668956717"
Nov 04 03:40:27.912133 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:27.912097    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30200904-1724-4ddd-aed5-2e9b612c26e9" path="/var/lib/kubelet/pods/30200904-1724-4ddd-aed5-2e9b612c26e9/volumes"
Nov 04 03:40:27.912615 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:27.912599    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="934b3519-cc81-4ea9-b8bd-9e962e9fcacb" path="/var/lib/kubelet/pods/934b3519-cc81-4ea9-b8bd-9e962e9fcacb/volumes"
Nov 04 03:40:27.913031 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:27.913018    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="efc6e068-c4fc-4559-a531-d62af03cbc9b" path="/var/lib/kubelet/pods/efc6e068-c4fc-4559-a531-d62af03cbc9b/volumes"
Nov 04 03:40:28.730027 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:40:28.729876    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:40:32.961759 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:40:32.961727    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:40:58.730178 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:40:58.730134    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:41:28.732293 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:41:28.732240    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:41:33.423775 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:33.423739    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"]
Nov 04 03:41:38.424457 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:38.424395    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c" podUID="811373cd-381a-4937-9b9c-594a56dc96d3" containerName="payload-processing" containerID="cri-o://62291a5826018042d05981ee52a49db56f81daf2594973d157fa5374e14f51ca" gracePeriod=25
Nov 04 03:41:38.688241 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:38.688181    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:41:38.847423 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:38.847393    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/811373cd-381a-4937-9b9c-594a56dc96d3-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/811373cd-381a-4937-9b9c-594a56dc96d3-routing-overlay-volume\") pod \"811373cd-381a-4937-9b9c-594a56dc96d3\" (UID: \"811373cd-381a-4937-9b9c-594a56dc96d3\") "
Nov 04 03:41:38.847586 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:38.847441    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/811373cd-381a-4937-9b9c-594a56dc96d3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/811373cd-381a-4937-9b9c-594a56dc96d3-plugins-config-volume\") pod \"811373cd-381a-4937-9b9c-594a56dc96d3\" (UID: \"811373cd-381a-4937-9b9c-594a56dc96d3\") "
Nov 04 03:41:38.847586 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:38.847514    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/811373cd-381a-4937-9b9c-594a56dc96d3-kube-api-access-x76zj\" (UniqueName: \"kubernetes.io/projected/811373cd-381a-4937-9b9c-594a56dc96d3-kube-api-access-x76zj\") pod \"811373cd-381a-4937-9b9c-594a56dc96d3\" (UID: \"811373cd-381a-4937-9b9c-594a56dc96d3\") "
Nov 04 03:41:38.847762 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:38.847650    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/811373cd-381a-4937-9b9c-594a56dc96d3-provider-credentials\" (UniqueName: \"kubernetes.io/projected/811373cd-381a-4937-9b9c-594a56dc96d3-provider-credentials\") pod \"811373cd-381a-4937-9b9c-594a56dc96d3\" (UID: \"811373cd-381a-4937-9b9c-594a56dc96d3\") "
Nov 04 03:41:38.847881 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:38.847821    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/811373cd-381a-4937-9b9c-594a56dc96d3-routing-overlay-volume" pod "811373cd-381a-4937-9b9c-594a56dc96d3" (UID: "811373cd-381a-4937-9b9c-594a56dc96d3"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:41:38.847881 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:38.847875    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/811373cd-381a-4937-9b9c-594a56dc96d3-plugins-config-volume" pod "811373cd-381a-4937-9b9c-594a56dc96d3" (UID: "811373cd-381a-4937-9b9c-594a56dc96d3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:41:38.848297 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:38.848250    4521 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/811373cd-381a-4937-9b9c-594a56dc96d3-routing-overlay-volume\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:38.848297 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:38.848279    4521 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/811373cd-381a-4937-9b9c-594a56dc96d3-plugins-config-volume\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:38.849891 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:38.849867    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/811373cd-381a-4937-9b9c-594a56dc96d3-provider-credentials" pod "811373cd-381a-4937-9b9c-594a56dc96d3" (UID: "811373cd-381a-4937-9b9c-594a56dc96d3"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:41:38.850027 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:38.850011    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/811373cd-381a-4937-9b9c-594a56dc96d3-kube-api-access-x76zj" pod "811373cd-381a-4937-9b9c-594a56dc96d3" (UID: "811373cd-381a-4937-9b9c-594a56dc96d3"). InnerVolumeSpecName "kube-api-access-x76zj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:41:38.949457 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:38.949362    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x76zj\" (UniqueName: \"kubernetes.io/projected/811373cd-381a-4937-9b9c-594a56dc96d3-kube-api-access-x76zj\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:38.949457 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:38.949396    4521 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/811373cd-381a-4937-9b9c-594a56dc96d3-provider-credentials\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:39.416652 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:39.416616    4521 generic.go:356] "Generic (PLEG): container finished" podID="811373cd-381a-4937-9b9c-594a56dc96d3" containerID="62291a5826018042d05981ee52a49db56f81daf2594973d157fa5374e14f51ca" exitCode=0
Nov 04 03:41:39.416881 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:39.416671    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"
Nov 04 03:41:39.416881 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:39.416702    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c" event={"ID":"811373cd-381a-4937-9b9c-594a56dc96d3","Type":"ContainerDied","Data":"62291a5826018042d05981ee52a49db56f81daf2594973d157fa5374e14f51ca"}
Nov 04 03:41:39.416881 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:39.416761    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c" event={"ID":"811373cd-381a-4937-9b9c-594a56dc96d3","Type":"ContainerDied","Data":"1c90ce5e39f22f302d23a6699aa60ae68957c3f597054d3be502b138fde96ec3"}
Nov 04 03:41:39.416881 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:39.416784    4521 scope.go:122] "RemoveContainer" containerID="62291a5826018042d05981ee52a49db56f81daf2594973d157fa5374e14f51ca"
Nov 04 03:41:39.427286 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:39.426967    4521 scope.go:122] "RemoveContainer" containerID="62291a5826018042d05981ee52a49db56f81daf2594973d157fa5374e14f51ca"
Nov 04 03:41:39.427286 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:41:39.427232    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62291a5826018042d05981ee52a49db56f81daf2594973d157fa5374e14f51ca\": container with ID starting with 62291a5826018042d05981ee52a49db56f81daf2594973d157fa5374e14f51ca not found: ID does not exist" containerID="62291a5826018042d05981ee52a49db56f81daf2594973d157fa5374e14f51ca"
Nov 04 03:41:39.427286 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:39.427261    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62291a5826018042d05981ee52a49db56f81daf2594973d157fa5374e14f51ca"} err="failed to get container status \"62291a5826018042d05981ee52a49db56f81daf2594973d157fa5374e14f51ca\": rpc error: code = NotFound desc = could not find container \"62291a5826018042d05981ee52a49db56f81daf2594973d157fa5374e14f51ca\": container with ID starting with 62291a5826018042d05981ee52a49db56f81daf2594973d157fa5374e14f51ca not found: ID does not exist"
Nov 04 03:41:39.584134 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:39.584100    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"]
Nov 04 03:41:39.591134 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:39.591103    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-59c311/payload-processing-external-model-e2e-extmod-b-59c311-75f72z95c"]
Nov 04 03:41:39.915584 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:39.915530    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="811373cd-381a-4937-9b9c-594a56dc96d3" path="/var/lib/kubelet/pods/811373cd-381a-4937-9b9c-594a56dc96d3/volumes"
Nov 04 03:41:45.143042 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.143000    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"]
Nov 04 03:41:45.143873 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.143828    4521 cpu_manager.go:414] "containerMap: removing container" podUID="30200904-1724-4ddd-aed5-2e9b612c26e9" containerName="istio-proxy"
Nov 04 03:41:45.143873 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.143867    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30200904-1724-4ddd-aed5-2e9b612c26e9" containerName="istio-proxy"
Nov 04 03:41:45.144065 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.143880    4521 cpu_manager.go:414] "containerMap: removing container" podUID="934b3519-cc81-4ea9-b8bd-9e962e9fcacb" containerName="istio-proxy"
Nov 04 03:41:45.144065 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.143888    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="934b3519-cc81-4ea9-b8bd-9e962e9fcacb" containerName="istio-proxy"
Nov 04 03:41:45.144065 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.143938    4521 cpu_manager.go:414] "containerMap: removing container" podUID="811373cd-381a-4937-9b9c-594a56dc96d3" containerName="payload-processing"
Nov 04 03:41:45.144065 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.143949    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="811373cd-381a-4937-9b9c-594a56dc96d3" containerName="payload-processing"
Nov 04 03:41:45.144065 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.143969    4521 cpu_manager.go:414] "containerMap: removing container" podUID="efc6e068-c4fc-4559-a531-d62af03cbc9b" containerName="istio-proxy"
Nov 04 03:41:45.144065 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.143977    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efc6e068-c4fc-4559-a531-d62af03cbc9b" containerName="istio-proxy"
Nov 04 03:41:45.144352 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.144157    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="efc6e068-c4fc-4559-a531-d62af03cbc9b" containerName="istio-proxy"
Nov 04 03:41:45.144352 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.144176    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="30200904-1724-4ddd-aed5-2e9b612c26e9" containerName="istio-proxy"
Nov 04 03:41:45.144352 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.144187    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="934b3519-cc81-4ea9-b8bd-9e962e9fcacb" containerName="istio-proxy"
Nov 04 03:41:45.144352 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.144199    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="811373cd-381a-4937-9b9c-594a56dc96d3" containerName="payload-processing"
Nov 04 03:41:45.149021 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.148997    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:41:45.206691 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.206660    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"]
Nov 04 03:41:45.231422 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.231392    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpjjg\" (UniqueName: \"kubernetes.io/projected/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-kube-api-access-tpjjg\") pod \"payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk\" (UID: \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\") " pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:41:45.231563 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.231447    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk\" (UID: \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\") " pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:41:45.231563 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.231474    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk\" (UID: \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\") " pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:41:45.231714 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.231684    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk\" (UID: \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\") " pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:41:45.253563 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.253538    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-951893\"/\"agc-cred-40eeab87dfd5\""
Nov 04 03:41:45.300189 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.300153    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-951893\"/\"kube-root-ca.crt\""
Nov 04 03:41:45.301207 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.300581    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-951893\"/\"openshift-service-ca.crt\""
Nov 04 03:41:45.305411 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.305388    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-951893\"/\"routing-overlay\""
Nov 04 03:41:45.305715 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.305690    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-951893\"/\"payload-processing-external-model-plugins-e2e-extmod-b-951893\""
Nov 04 03:41:45.332546 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.332511    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpjjg\" (UniqueName: \"kubernetes.io/projected/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-kube-api-access-tpjjg\") pod \"payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk\" (UID: \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\") " pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:41:45.332671 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.332569    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk\" (UID: \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\") " pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:41:45.332671 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.332593    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk\" (UID: \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\") " pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:41:45.332671 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.332661    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk\" (UID: \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\") " pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:41:45.333275 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.333245    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk\" (UID: \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\") " pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:41:45.333387 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.333364    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk\" (UID: \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\") " pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:41:45.335087 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.335069    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk\" (UID: \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\") " pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:41:45.394507 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.394431    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpjjg\" (UniqueName: \"kubernetes.io/projected/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-kube-api-access-tpjjg\") pod \"payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk\" (UID: \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\") " pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:41:45.463209 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.463173    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:41:45.846415 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:45.846382    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"]
Nov 04 03:41:45.848107 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:41:45.848073    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c3b45fd_7af2_4168_bc73_2a8e46e8d1e1.slice/crio-8c948ba0684382cddf558fe10818b5fb679731a39f7c1930e9e03c8a7333e70e WatchSource:0}: Error finding container 8c948ba0684382cddf558fe10818b5fb679731a39f7c1930e9e03c8a7333e70e: Status 404 returned error can't find the container with id 8c948ba0684382cddf558fe10818b5fb679731a39f7c1930e9e03c8a7333e70e
Nov 04 03:41:46.461442 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:46.461391    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk" event={"ID":"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1","Type":"ContainerStarted","Data":"18c6552a86ab1a34b3d3a98db384eb39910f74aed3f80a460275601ae3b6e494"}
Nov 04 03:41:46.461949 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:46.461482    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk" event={"ID":"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1","Type":"ContainerStarted","Data":"8c948ba0684382cddf558fe10818b5fb679731a39f7c1930e9e03c8a7333e70e"}
Nov 04 03:41:46.461949 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:46.461511    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:41:46.580439 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:46.580380    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk" podStartSLOduration=1.580366776 podStartE2EDuration="1.580366776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:41:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:41:46.577983979 +0000 UTC m=+1472.782584299" watchObservedRunningTime="2026-11-04 03:41:46.580366776 +0000 UTC m=+1472.784967078"
Nov 04 03:41:51.780645 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:51.780609    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"]
Nov 04 03:41:56.781057 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:56.780988    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn" podUID="3483c5f0-468c-4b53-8efb-77788eae77a8" containerName="payload-processing" containerID="cri-o://16d230f15333cff75956e8eca7d6d2eacc5a34465f399dca148ddfa043741a5d" gracePeriod=25
Nov 04 03:41:56.823937 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:41:56.823908    4521 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3483c5f0_468c_4b53_8efb_77788eae77a8.slice/crio-16d230f15333cff75956e8eca7d6d2eacc5a34465f399dca148ddfa043741a5d.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 03:41:57.046002 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.045944    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"
Nov 04 03:41:57.187487 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.187444    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3483c5f0-468c-4b53-8efb-77788eae77a8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3483c5f0-468c-4b53-8efb-77788eae77a8-plugins-config-volume\") pod \"3483c5f0-468c-4b53-8efb-77788eae77a8\" (UID: \"3483c5f0-468c-4b53-8efb-77788eae77a8\") "
Nov 04 03:41:57.187641 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.187547    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3483c5f0-468c-4b53-8efb-77788eae77a8-kube-api-access-bz62d\" (UniqueName: \"kubernetes.io/projected/3483c5f0-468c-4b53-8efb-77788eae77a8-kube-api-access-bz62d\") pod \"3483c5f0-468c-4b53-8efb-77788eae77a8\" (UID: \"3483c5f0-468c-4b53-8efb-77788eae77a8\") "
Nov 04 03:41:57.187863 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.187821    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3483c5f0-468c-4b53-8efb-77788eae77a8-plugins-config-volume" pod "3483c5f0-468c-4b53-8efb-77788eae77a8" (UID: "3483c5f0-468c-4b53-8efb-77788eae77a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:41:57.189767 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.189737    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3483c5f0-468c-4b53-8efb-77788eae77a8-kube-api-access-bz62d" pod "3483c5f0-468c-4b53-8efb-77788eae77a8" (UID: "3483c5f0-468c-4b53-8efb-77788eae77a8"). InnerVolumeSpecName "kube-api-access-bz62d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:41:57.289288 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.289248    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz62d\" (UniqueName: \"kubernetes.io/projected/3483c5f0-468c-4b53-8efb-77788eae77a8-kube-api-access-bz62d\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:57.289288 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.289286    4521 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3483c5f0-468c-4b53-8efb-77788eae77a8-plugins-config-volume\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:57.471775 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.471703    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:41:57.528854 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.528816    4521 generic.go:356] "Generic (PLEG): container finished" podID="3483c5f0-468c-4b53-8efb-77788eae77a8" containerID="16d230f15333cff75956e8eca7d6d2eacc5a34465f399dca148ddfa043741a5d" exitCode=0
Nov 04 03:41:57.529122 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.528869    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"
Nov 04 03:41:57.529122 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.528901    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn" event={"ID":"3483c5f0-468c-4b53-8efb-77788eae77a8","Type":"ContainerDied","Data":"16d230f15333cff75956e8eca7d6d2eacc5a34465f399dca148ddfa043741a5d"}
Nov 04 03:41:57.529122 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.528963    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn" event={"ID":"3483c5f0-468c-4b53-8efb-77788eae77a8","Type":"ContainerDied","Data":"5fa9206222cf3f8d7e81fc4d16aeace3be0fa146c7a43ccb91f38c01377ff29a"}
Nov 04 03:41:57.529122 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.528993    4521 scope.go:122] "RemoveContainer" containerID="16d230f15333cff75956e8eca7d6d2eacc5a34465f399dca148ddfa043741a5d"
Nov 04 03:41:57.538796 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.538775    4521 scope.go:122] "RemoveContainer" containerID="16d230f15333cff75956e8eca7d6d2eacc5a34465f399dca148ddfa043741a5d"
Nov 04 03:41:57.539109 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:41:57.539087    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16d230f15333cff75956e8eca7d6d2eacc5a34465f399dca148ddfa043741a5d\": container with ID starting with 16d230f15333cff75956e8eca7d6d2eacc5a34465f399dca148ddfa043741a5d not found: ID does not exist" containerID="16d230f15333cff75956e8eca7d6d2eacc5a34465f399dca148ddfa043741a5d"
Nov 04 03:41:57.539173 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.539119    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16d230f15333cff75956e8eca7d6d2eacc5a34465f399dca148ddfa043741a5d"} err="failed to get container status \"16d230f15333cff75956e8eca7d6d2eacc5a34465f399dca148ddfa043741a5d\": rpc error: code = NotFound desc = could not find container \"16d230f15333cff75956e8eca7d6d2eacc5a34465f399dca148ddfa043741a5d\": container with ID starting with 16d230f15333cff75956e8eca7d6d2eacc5a34465f399dca148ddfa043741a5d not found: ID does not exist"
Nov 04 03:41:57.703339 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.703306    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"]
Nov 04 03:41:57.707778 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.707753    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-1bf4bf-6699d6c4bd-5rfjn"]
Nov 04 03:41:57.915188 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:41:57.915136    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3483c5f0-468c-4b53-8efb-77788eae77a8" path="/var/lib/kubelet/pods/3483c5f0-468c-4b53-8efb-77788eae77a8/volumes"
Nov 04 03:41:58.733525 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:41:58.733406    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:42:28.730576 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:42:28.730536    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:42:46.603327 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:46.603286    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"]
Nov 04 03:42:46.652439 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:46.652404    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"]
Nov 04 03:42:46.923497 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:46.923410    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"]
Nov 04 03:42:51.710691 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:51.710655    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"]
Nov 04 03:42:51.924494 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:51.924452    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv" podUID="95615e76-0669-492a-b8a5-9b48b7fb1b15" containerName="payload-processing" containerID="cri-o://764d07a8060e82091d9dd08cd58fe5294e1fce4277ae76547941f2148792412e" gracePeriod=25
Nov 04 03:42:52.190000 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:52.189973    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"
Nov 04 03:42:52.338219 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:52.338116    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95615e76-0669-492a-b8a5-9b48b7fb1b15-kube-api-access-5ztpz\" (UniqueName: \"kubernetes.io/projected/95615e76-0669-492a-b8a5-9b48b7fb1b15-kube-api-access-5ztpz\") pod \"95615e76-0669-492a-b8a5-9b48b7fb1b15\" (UID: \"95615e76-0669-492a-b8a5-9b48b7fb1b15\") "
Nov 04 03:42:52.338411 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:52.338348    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95615e76-0669-492a-b8a5-9b48b7fb1b15-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95615e76-0669-492a-b8a5-9b48b7fb1b15-plugins-config-volume\") pod \"95615e76-0669-492a-b8a5-9b48b7fb1b15\" (UID: \"95615e76-0669-492a-b8a5-9b48b7fb1b15\") "
Nov 04 03:42:52.338805 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:52.338765    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95615e76-0669-492a-b8a5-9b48b7fb1b15-plugins-config-volume" pod "95615e76-0669-492a-b8a5-9b48b7fb1b15" (UID: "95615e76-0669-492a-b8a5-9b48b7fb1b15"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:42:52.340555 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:52.340520    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95615e76-0669-492a-b8a5-9b48b7fb1b15-kube-api-access-5ztpz" pod "95615e76-0669-492a-b8a5-9b48b7fb1b15" (UID: "95615e76-0669-492a-b8a5-9b48b7fb1b15"). InnerVolumeSpecName "kube-api-access-5ztpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:52.440386 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:52.440353    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ztpz\" (UniqueName: \"kubernetes.io/projected/95615e76-0669-492a-b8a5-9b48b7fb1b15-kube-api-access-5ztpz\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:52.440386 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:52.440381    4521 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95615e76-0669-492a-b8a5-9b48b7fb1b15-plugins-config-volume\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:52.856695 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:52.856656    4521 generic.go:356] "Generic (PLEG): container finished" podID="95615e76-0669-492a-b8a5-9b48b7fb1b15" containerID="764d07a8060e82091d9dd08cd58fe5294e1fce4277ae76547941f2148792412e" exitCode=0
Nov 04 03:42:52.857201 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:52.856701    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"
Nov 04 03:42:52.857201 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:52.856742    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv" event={"ID":"95615e76-0669-492a-b8a5-9b48b7fb1b15","Type":"ContainerDied","Data":"764d07a8060e82091d9dd08cd58fe5294e1fce4277ae76547941f2148792412e"}
Nov 04 03:42:52.857201 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:52.856806    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv" event={"ID":"95615e76-0669-492a-b8a5-9b48b7fb1b15","Type":"ContainerDied","Data":"97ca198a055d74ed696ba022d70db553b301d9f4c985b872076d5b9cbbb8fdd0"}
Nov 04 03:42:52.857201 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:52.856829    4521 scope.go:122] "RemoveContainer" containerID="764d07a8060e82091d9dd08cd58fe5294e1fce4277ae76547941f2148792412e"
Nov 04 03:42:52.867086 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:52.867069    4521 scope.go:122] "RemoveContainer" containerID="764d07a8060e82091d9dd08cd58fe5294e1fce4277ae76547941f2148792412e"
Nov 04 03:42:52.867368 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:42:52.867349    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"764d07a8060e82091d9dd08cd58fe5294e1fce4277ae76547941f2148792412e\": container with ID starting with 764d07a8060e82091d9dd08cd58fe5294e1fce4277ae76547941f2148792412e not found: ID does not exist" containerID="764d07a8060e82091d9dd08cd58fe5294e1fce4277ae76547941f2148792412e"
Nov 04 03:42:52.867417 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:52.867377    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"764d07a8060e82091d9dd08cd58fe5294e1fce4277ae76547941f2148792412e"} err="failed to get container status \"764d07a8060e82091d9dd08cd58fe5294e1fce4277ae76547941f2148792412e\": rpc error: code = NotFound desc = could not find container \"764d07a8060e82091d9dd08cd58fe5294e1fce4277ae76547941f2148792412e\": container with ID starting with 764d07a8060e82091d9dd08cd58fe5294e1fce4277ae76547941f2148792412e not found: ID does not exist"
Nov 04 03:42:53.025203 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:53.025169    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"]
Nov 04 03:42:53.075591 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:53.075556    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-59c311-6987c8b78-vpbrv"]
Nov 04 03:42:53.912615 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:53.912582    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95615e76-0669-492a-b8a5-9b48b7fb1b15" path="/var/lib/kubelet/pods/95615e76-0669-492a-b8a5-9b48b7fb1b15/volumes"
Nov 04 03:42:54.411442 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:54.411405    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"]
Nov 04 03:42:56.711390 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:56.711352    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk" podUID="1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1" containerName="payload-processing" containerID="cri-o://18c6552a86ab1a34b3d3a98db384eb39910f74aed3f80a460275601ae3b6e494" gracePeriod=25
Nov 04 03:42:56.894453 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:56.894317    4521 generic.go:356] "Generic (PLEG): container finished" podID="1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1" containerID="18c6552a86ab1a34b3d3a98db384eb39910f74aed3f80a460275601ae3b6e494" exitCode=0
Nov 04 03:42:56.894453 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:56.894395    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk" event={"ID":"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1","Type":"ContainerDied","Data":"18c6552a86ab1a34b3d3a98db384eb39910f74aed3f80a460275601ae3b6e494"}
Nov 04 03:42:56.968306 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:56.968248    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:42:56.993177 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:56.993150    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-routing-overlay-volume\") pod \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\" (UID: \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\") "
Nov 04 03:42:56.993309 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:56.993195    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-provider-credentials\" (UniqueName: \"kubernetes.io/projected/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-provider-credentials\") pod \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\" (UID: \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\") "
Nov 04 03:42:56.993309 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:56.993221    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-kube-api-access-tpjjg\" (UniqueName: \"kubernetes.io/projected/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-kube-api-access-tpjjg\") pod \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\" (UID: \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\") "
Nov 04 03:42:56.993309 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:56.993305    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-plugins-config-volume\") pod \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\" (UID: \"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1\") "
Nov 04 03:42:56.993492 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:56.993462    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-routing-overlay-volume" pod "1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1" (UID: "1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:42:56.993771 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:56.993745    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-plugins-config-volume" pod "1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1" (UID: "1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:42:56.994075 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:56.994004    4521 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-plugins-config-volume\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:56.994075 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:56.994030    4521 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-routing-overlay-volume\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:56.995553 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:56.995526    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-provider-credentials" pod "1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1" (UID: "1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:56.995823 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:56.995805    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-kube-api-access-tpjjg" pod "1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1" (UID: "1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1"). InnerVolumeSpecName "kube-api-access-tpjjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:57.095480 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.095445    4521 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-provider-credentials\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:57.095480 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.095472    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpjjg\" (UniqueName: \"kubernetes.io/projected/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1-kube-api-access-tpjjg\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:57.276504 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.276469    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"]
Nov 04 03:42:57.276686 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.276665    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj" podUID="abf30d8d-9a93-4298-9ee3-92887e638513" containerName="maas-api" containerID="cri-o://a127b528f942d653001126063ff963026325c358fea02e95d9511bea84a12fd1" gracePeriod=30
Nov 04 03:42:57.514796 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.514768    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:42:57.601755 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.601670    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/abf30d8d-9a93-4298-9ee3-92887e638513-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf30d8d-9a93-4298-9ee3-92887e638513-maas-api-tls\") pod \"abf30d8d-9a93-4298-9ee3-92887e638513\" (UID: \"abf30d8d-9a93-4298-9ee3-92887e638513\") "
Nov 04 03:42:57.601755 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.601722    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/abf30d8d-9a93-4298-9ee3-92887e638513-kube-api-access-zjq8r\" (UniqueName: \"kubernetes.io/projected/abf30d8d-9a93-4298-9ee3-92887e638513-kube-api-access-zjq8r\") pod \"abf30d8d-9a93-4298-9ee3-92887e638513\" (UID: \"abf30d8d-9a93-4298-9ee3-92887e638513\") "
Nov 04 03:42:57.602006 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.601758    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/abf30d8d-9a93-4298-9ee3-92887e638513-metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf30d8d-9a93-4298-9ee3-92887e638513-metrics-certs\") pod \"abf30d8d-9a93-4298-9ee3-92887e638513\" (UID: \"abf30d8d-9a93-4298-9ee3-92887e638513\") "
Nov 04 03:42:57.602006 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.601812    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/abf30d8d-9a93-4298-9ee3-92887e638513-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf30d8d-9a93-4298-9ee3-92887e638513-openshift-service-ca\") pod \"abf30d8d-9a93-4298-9ee3-92887e638513\" (UID: \"abf30d8d-9a93-4298-9ee3-92887e638513\") "
Nov 04 03:42:57.602285 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.602262    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abf30d8d-9a93-4298-9ee3-92887e638513-openshift-service-ca" pod "abf30d8d-9a93-4298-9ee3-92887e638513" (UID: "abf30d8d-9a93-4298-9ee3-92887e638513"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:42:57.603908 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.603891    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abf30d8d-9a93-4298-9ee3-92887e638513-metrics-certs" pod "abf30d8d-9a93-4298-9ee3-92887e638513" (UID: "abf30d8d-9a93-4298-9ee3-92887e638513"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:42:57.604393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.604370    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abf30d8d-9a93-4298-9ee3-92887e638513-maas-api-tls" pod "abf30d8d-9a93-4298-9ee3-92887e638513" (UID: "abf30d8d-9a93-4298-9ee3-92887e638513"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:42:57.604440 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.604397    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abf30d8d-9a93-4298-9ee3-92887e638513-kube-api-access-zjq8r" pod "abf30d8d-9a93-4298-9ee3-92887e638513" (UID: "abf30d8d-9a93-4298-9ee3-92887e638513"). InnerVolumeSpecName "kube-api-access-zjq8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:57.705271 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.703760    4521 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf30d8d-9a93-4298-9ee3-92887e638513-maas-api-tls\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:57.705271 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.703796    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjq8r\" (UniqueName: \"kubernetes.io/projected/abf30d8d-9a93-4298-9ee3-92887e638513-kube-api-access-zjq8r\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:57.705271 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.703812    4521 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf30d8d-9a93-4298-9ee3-92887e638513-metrics-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:57.705271 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.703826    4521 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf30d8d-9a93-4298-9ee3-92887e638513-openshift-service-ca\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:57.901096 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.901012    4521 generic.go:356] "Generic (PLEG): container finished" podID="abf30d8d-9a93-4298-9ee3-92887e638513" containerID="a127b528f942d653001126063ff963026325c358fea02e95d9511bea84a12fd1" exitCode=0
Nov 04 03:42:57.901096 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.901058    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"
Nov 04 03:42:57.901574 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.901095    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj" event={"ID":"abf30d8d-9a93-4298-9ee3-92887e638513","Type":"ContainerDied","Data":"a127b528f942d653001126063ff963026325c358fea02e95d9511bea84a12fd1"}
Nov 04 03:42:57.901574 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.901151    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj" event={"ID":"abf30d8d-9a93-4298-9ee3-92887e638513","Type":"ContainerDied","Data":"1f29027130477ab0304be8dc79de580ed38b799569b7854071e6f66b8dec5444"}
Nov 04 03:42:57.901574 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.901174    4521 scope.go:122] "RemoveContainer" containerID="a127b528f942d653001126063ff963026325c358fea02e95d9511bea84a12fd1"
Nov 04 03:42:57.902719 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.902695    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk" event={"ID":"1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1","Type":"ContainerDied","Data":"8c948ba0684382cddf558fe10818b5fb679731a39f7c1930e9e03c8a7333e70e"}
Nov 04 03:42:57.902830 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.902700    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"
Nov 04 03:42:57.913299 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.913237    4521 scope.go:122] "RemoveContainer" containerID="a127b528f942d653001126063ff963026325c358fea02e95d9511bea84a12fd1"
Nov 04 03:42:57.913714 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:42:57.913534    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a127b528f942d653001126063ff963026325c358fea02e95d9511bea84a12fd1\": container with ID starting with a127b528f942d653001126063ff963026325c358fea02e95d9511bea84a12fd1 not found: ID does not exist" containerID="a127b528f942d653001126063ff963026325c358fea02e95d9511bea84a12fd1"
Nov 04 03:42:57.913714 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.913559    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a127b528f942d653001126063ff963026325c358fea02e95d9511bea84a12fd1"} err="failed to get container status \"a127b528f942d653001126063ff963026325c358fea02e95d9511bea84a12fd1\": rpc error: code = NotFound desc = could not find container \"a127b528f942d653001126063ff963026325c358fea02e95d9511bea84a12fd1\": container with ID starting with a127b528f942d653001126063ff963026325c358fea02e95d9511bea84a12fd1 not found: ID does not exist"
Nov 04 03:42:57.913714 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:57.913577    4521 scope.go:122] "RemoveContainer" containerID="18c6552a86ab1a34b3d3a98db384eb39910f74aed3f80a460275601ae3b6e494"
Nov 04 03:42:58.072681 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:58.072636    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"]
Nov 04 03:42:58.119715 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:58.119675    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-951893/payload-processing-external-model-e2e-extmod-b-951893-7494m9kdk"]
Nov 04 03:42:58.251885 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:58.251777    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"]
Nov 04 03:42:58.256341 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:58.256313    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-59c311-6f67f4ddd9-h78zj"]
Nov 04 03:42:58.730602 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:42:58.730558    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:42:59.412063 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:59.412019    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx" podUID="c8e3aedf-25cb-4002-8fd3-70b762747e3e" containerName="payload-pre-processing" containerID="cri-o://772681783b2eedbc2486ed64096150bf00981774035300a17a0507db9d5de48f" gracePeriod=25
Nov 04 03:42:59.912392 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:59.912359    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1" path="/var/lib/kubelet/pods/1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1/volumes"
Nov 04 03:42:59.912862 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:42:59.912820    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="abf30d8d-9a93-4298-9ee3-92887e638513" path="/var/lib/kubelet/pods/abf30d8d-9a93-4298-9ee3-92887e638513/volumes"
Nov 04 03:43:01.628268 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:01.628226    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b" podUID="672c902d-f63e-42e6-b1e2-407d6e148c04" containerName="main" containerID="cri-o://414c8851e7b8a42353ba5543e8b87b93143a1a1418d981dac3578d71cad2f24a" gracePeriod=45
Nov 04 03:43:01.673366 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:01.673332    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9" podUID="69b690e9-f65d-42c9-9ca3-aea2ec55f30a" containerName="main" containerID="cri-o://fa84ef26809e229ef196c3c3dfe502d0452e92f040ec7b2189350095d4a78533" gracePeriod=45
Nov 04 03:43:01.934740 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:01.934702    4521 generic.go:356] "Generic (PLEG): container finished" podID="672c902d-f63e-42e6-b1e2-407d6e148c04" containerID="414c8851e7b8a42353ba5543e8b87b93143a1a1418d981dac3578d71cad2f24a" exitCode=0
Nov 04 03:43:01.935080 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:01.935046    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b" event={"ID":"672c902d-f63e-42e6-b1e2-407d6e148c04","Type":"ContainerDied","Data":"414c8851e7b8a42353ba5543e8b87b93143a1a1418d981dac3578d71cad2f24a"}
Nov 04 03:43:01.935211 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:01.935119    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b" event={"ID":"672c902d-f63e-42e6-b1e2-407d6e148c04","Type":"ContainerDied","Data":"5e7437da821733036a8737696f643c9423b14f63cf441b0e6f908e514a272a4f"}
Nov 04 03:43:01.935211 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:01.935138    4521 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e7437da821733036a8737696f643c9423b14f63cf441b0e6f908e514a272a4f"
Nov 04 03:43:01.936528 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:01.936507    4521 generic.go:356] "Generic (PLEG): container finished" podID="69b690e9-f65d-42c9-9ca3-aea2ec55f30a" containerID="fa84ef26809e229ef196c3c3dfe502d0452e92f040ec7b2189350095d4a78533" exitCode=0
Nov 04 03:43:01.936649 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:01.936556    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9" event={"ID":"69b690e9-f65d-42c9-9ca3-aea2ec55f30a","Type":"ContainerDied","Data":"fa84ef26809e229ef196c3c3dfe502d0452e92f040ec7b2189350095d4a78533"}
Nov 04 03:43:01.955354 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:01.955334    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:43:01.958698 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:01.958680    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:43:02.052557 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.052518    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-dshm\") pod \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") "
Nov 04 03:43:02.052743 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.052576    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-tmp-dir\") pod \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") "
Nov 04 03:43:02.052743 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.052644    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/672c902d-f63e-42e6-b1e2-407d6e148c04-tls-certs\" (UniqueName: \"kubernetes.io/secret/672c902d-f63e-42e6-b1e2-407d6e148c04-tls-certs\") pod \"672c902d-f63e-42e6-b1e2-407d6e148c04\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") "
Nov 04 03:43:02.052743 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.052695    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-tls-certs\" (UniqueName: \"kubernetes.io/secret/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-tls-certs\") pod \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") "
Nov 04 03:43:02.052945 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.052744    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-model-cache\") pod \"672c902d-f63e-42e6-b1e2-407d6e148c04\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") "
Nov 04 03:43:02.052945 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.052800    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-model-cache\") pod \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") "
Nov 04 03:43:02.052945 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.052874    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/672c902d-f63e-42e6-b1e2-407d6e148c04-kube-api-access-nlxjb\" (UniqueName: \"kubernetes.io/projected/672c902d-f63e-42e6-b1e2-407d6e148c04-kube-api-access-nlxjb\") pod \"672c902d-f63e-42e6-b1e2-407d6e148c04\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") "
Nov 04 03:43:02.052945 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.052921    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-tmp-dir\") pod \"672c902d-f63e-42e6-b1e2-407d6e148c04\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") "
Nov 04 03:43:02.052945 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.052933    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-tmp-dir" pod "69b690e9-f65d-42c9-9ca3-aea2ec55f30a" (UID: "69b690e9-f65d-42c9-9ca3-aea2ec55f30a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:02.053201 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.052985    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-dshm\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-dshm\") pod \"672c902d-f63e-42e6-b1e2-407d6e148c04\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") "
Nov 04 03:43:02.053201 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.053044    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-home\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-home\") pod \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") "
Nov 04 03:43:02.053201 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.053085    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-home\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-home\") pod \"672c902d-f63e-42e6-b1e2-407d6e148c04\" (UID: \"672c902d-f63e-42e6-b1e2-407d6e148c04\") "
Nov 04 03:43:02.053201 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.053084    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-model-cache" pod "672c902d-f63e-42e6-b1e2-407d6e148c04" (UID: "672c902d-f63e-42e6-b1e2-407d6e148c04"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:02.053201 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.053133    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-kube-api-access-zcbm4\" (UniqueName: \"kubernetes.io/projected/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-kube-api-access-zcbm4\") pod \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\" (UID: \"69b690e9-f65d-42c9-9ca3-aea2ec55f30a\") "
Nov 04 03:43:02.053201 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.053155    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-model-cache" pod "69b690e9-f65d-42c9-9ca3-aea2ec55f30a" (UID: "69b690e9-f65d-42c9-9ca3-aea2ec55f30a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:02.053501 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.053390    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-tmp-dir" pod "672c902d-f63e-42e6-b1e2-407d6e148c04" (UID: "672c902d-f63e-42e6-b1e2-407d6e148c04"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:02.053501 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.053394    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-home" pod "69b690e9-f65d-42c9-9ca3-aea2ec55f30a" (UID: "69b690e9-f65d-42c9-9ca3-aea2ec55f30a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:02.053627 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.053575    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-home" pod "672c902d-f63e-42e6-b1e2-407d6e148c04" (UID: "672c902d-f63e-42e6-b1e2-407d6e148c04"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:02.053711 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.053689    4521 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-tmp-dir\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:02.053775 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.053719    4521 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-model-cache\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:02.053775 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.053734    4521 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-model-cache\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:02.053775 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.053748    4521 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-tmp-dir\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:02.053775 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.053762    4521 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-home\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:02.054000 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.053776    4521 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-home\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:02.055081 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.055053    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-dshm" pod "69b690e9-f65d-42c9-9ca3-aea2ec55f30a" (UID: "69b690e9-f65d-42c9-9ca3-aea2ec55f30a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:02.055171 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.055150    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/672c902d-f63e-42e6-b1e2-407d6e148c04-kube-api-access-nlxjb" pod "672c902d-f63e-42e6-b1e2-407d6e148c04" (UID: "672c902d-f63e-42e6-b1e2-407d6e148c04"). InnerVolumeSpecName "kube-api-access-nlxjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:02.055356 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.055339    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-tls-certs" pod "69b690e9-f65d-42c9-9ca3-aea2ec55f30a" (UID: "69b690e9-f65d-42c9-9ca3-aea2ec55f30a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:43:02.055572 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.055557    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-dshm" pod "672c902d-f63e-42e6-b1e2-407d6e148c04" (UID: "672c902d-f63e-42e6-b1e2-407d6e148c04"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:02.055654 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.055601    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-kube-api-access-zcbm4" pod "69b690e9-f65d-42c9-9ca3-aea2ec55f30a" (UID: "69b690e9-f65d-42c9-9ca3-aea2ec55f30a"). InnerVolumeSpecName "kube-api-access-zcbm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:02.055874 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.055831    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/672c902d-f63e-42e6-b1e2-407d6e148c04-tls-certs" pod "672c902d-f63e-42e6-b1e2-407d6e148c04" (UID: "672c902d-f63e-42e6-b1e2-407d6e148c04"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:43:02.154570 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.154532    4521 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-dshm\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:02.154570 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.154563    4521 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/672c902d-f63e-42e6-b1e2-407d6e148c04-tls-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:02.154570 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.154576    4521 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-tls-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:02.154814 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.154589    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlxjb\" (UniqueName: \"kubernetes.io/projected/672c902d-f63e-42e6-b1e2-407d6e148c04-kube-api-access-nlxjb\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:02.154814 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.154601    4521 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/672c902d-f63e-42e6-b1e2-407d6e148c04-dshm\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:02.154814 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.154613    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcbm4\" (UniqueName: \"kubernetes.io/projected/69b690e9-f65d-42c9-9ca3-aea2ec55f30a-kube-api-access-zcbm4\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:02.429016 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.428960    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx" podUID="c8e3aedf-25cb-4002-8fd3-70b762747e3e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:43:02.944076 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.944045    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"
Nov 04 03:43:02.944076 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.944063    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9" event={"ID":"69b690e9-f65d-42c9-9ca3-aea2ec55f30a","Type":"ContainerDied","Data":"5723e3668d3a9c5b665881dfccd5b8c565272b41768259c2a1155f081d6bda4a"}
Nov 04 03:43:02.944576 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.944113    4521 scope.go:122] "RemoveContainer" containerID="fa84ef26809e229ef196c3c3dfe502d0452e92f040ec7b2189350095d4a78533"
Nov 04 03:43:02.944576 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:02.944309    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"
Nov 04 03:43:03.119519 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:03.119483    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"]
Nov 04 03:43:03.126139 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:03.126114    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-50429e/premium-w4-50429e-kserve-6f484d58fc-vk72b"]
Nov 04 03:43:03.235904 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:03.235821    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"]
Nov 04 03:43:03.241887 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:03.241860    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-50429e/simulator-w4-50429e-kserve-67d765f84-bgkh9"]
Nov 04 03:43:03.911646 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:03.911612    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="672c902d-f63e-42e6-b1e2-407d6e148c04" path="/var/lib/kubelet/pods/672c902d-f63e-42e6-b1e2-407d6e148c04/volumes"
Nov 04 03:43:03.912065 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:03.912051    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="69b690e9-f65d-42c9-9ca3-aea2ec55f30a" path="/var/lib/kubelet/pods/69b690e9-f65d-42c9-9ca3-aea2ec55f30a/volumes"
Nov 04 03:43:08.350867 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:08.350462    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"]
Nov 04 03:43:12.428361 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:12.428315    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx" podUID="c8e3aedf-25cb-4002-8fd3-70b762747e3e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:43:14.195510 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:14.195479    4521 scope.go:122] "RemoveContainer" containerID="414c8851e7b8a42353ba5543e8b87b93143a1a1418d981dac3578d71cad2f24a"
Nov 04 03:43:19.671127 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:19.671063    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"
Nov 04 03:43:19.821143 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:19.821109    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8e3aedf-25cb-4002-8fd3-70b762747e3e-kube-api-access-58m7z\" (UniqueName: \"kubernetes.io/projected/c8e3aedf-25cb-4002-8fd3-70b762747e3e-kube-api-access-58m7z\") pod \"c8e3aedf-25cb-4002-8fd3-70b762747e3e\" (UID: \"c8e3aedf-25cb-4002-8fd3-70b762747e3e\") "
Nov 04 03:43:19.821326 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:19.821197    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8e3aedf-25cb-4002-8fd3-70b762747e3e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8e3aedf-25cb-4002-8fd3-70b762747e3e-plugins-config-volume\") pod \"c8e3aedf-25cb-4002-8fd3-70b762747e3e\" (UID: \"c8e3aedf-25cb-4002-8fd3-70b762747e3e\") "
Nov 04 03:43:19.821578 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:19.821553    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8e3aedf-25cb-4002-8fd3-70b762747e3e-plugins-config-volume" pod "c8e3aedf-25cb-4002-8fd3-70b762747e3e" (UID: "c8e3aedf-25cb-4002-8fd3-70b762747e3e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:43:19.823270 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:19.823249    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8e3aedf-25cb-4002-8fd3-70b762747e3e-kube-api-access-58m7z" pod "c8e3aedf-25cb-4002-8fd3-70b762747e3e" (UID: "c8e3aedf-25cb-4002-8fd3-70b762747e3e"). InnerVolumeSpecName "kube-api-access-58m7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:19.922122 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:19.922069    4521 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8e3aedf-25cb-4002-8fd3-70b762747e3e-plugins-config-volume\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:19.922122 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:19.922094    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58m7z\" (UniqueName: \"kubernetes.io/projected/c8e3aedf-25cb-4002-8fd3-70b762747e3e-kube-api-access-58m7z\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:20.094193 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:20.094157    4521 generic.go:356] "Generic (PLEG): container finished" podID="c8e3aedf-25cb-4002-8fd3-70b762747e3e" containerID="772681783b2eedbc2486ed64096150bf00981774035300a17a0507db9d5de48f" exitCode=0
Nov 04 03:43:20.094383 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:20.094207    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"
Nov 04 03:43:20.094383 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:20.094237    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx" event={"ID":"c8e3aedf-25cb-4002-8fd3-70b762747e3e","Type":"ContainerDied","Data":"772681783b2eedbc2486ed64096150bf00981774035300a17a0507db9d5de48f"}
Nov 04 03:43:20.094383 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:20.094292    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx" event={"ID":"c8e3aedf-25cb-4002-8fd3-70b762747e3e","Type":"ContainerDied","Data":"a1fd41d50084b9c366390515a3e8892b76620a34acba01b6eaa35f815a75394c"}
Nov 04 03:43:20.094383 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:20.094314    4521 scope.go:122] "RemoveContainer" containerID="772681783b2eedbc2486ed64096150bf00981774035300a17a0507db9d5de48f"
Nov 04 03:43:20.103357 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:20.103338    4521 scope.go:122] "RemoveContainer" containerID="772681783b2eedbc2486ed64096150bf00981774035300a17a0507db9d5de48f"
Nov 04 03:43:20.103637 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:43:20.103618    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"772681783b2eedbc2486ed64096150bf00981774035300a17a0507db9d5de48f\": container with ID starting with 772681783b2eedbc2486ed64096150bf00981774035300a17a0507db9d5de48f not found: ID does not exist" containerID="772681783b2eedbc2486ed64096150bf00981774035300a17a0507db9d5de48f"
Nov 04 03:43:20.103711 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:20.103643    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"772681783b2eedbc2486ed64096150bf00981774035300a17a0507db9d5de48f"} err="failed to get container status \"772681783b2eedbc2486ed64096150bf00981774035300a17a0507db9d5de48f\": rpc error: code = NotFound desc = could not find container \"772681783b2eedbc2486ed64096150bf00981774035300a17a0507db9d5de48f\": container with ID starting with 772681783b2eedbc2486ed64096150bf00981774035300a17a0507db9d5de48f not found: ID does not exist"
Nov 04 03:43:20.264685 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:20.264608    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"]
Nov 04 03:43:20.273967 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:20.273937    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-50429e-77f4c85c59-wphrx"]
Nov 04 03:43:21.923620 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:21.923584    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8e3aedf-25cb-4002-8fd3-70b762747e3e" path="/var/lib/kubelet/pods/c8e3aedf-25cb-4002-8fd3-70b762747e3e/volumes"
Nov 04 03:43:23.382853 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.382803    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv" podUID="c53728db-b02a-497b-9a90-7f47a026e656" containerName="main" containerID="cri-o://dc8e5c1527037e5c8860e31cb626bf5fa1362517adfdaa0401f05b973f06a29b" gracePeriod=45
Nov 04 03:43:23.644615 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.644561    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:43:23.770268 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.769089    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-home\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-home\") pod \"c53728db-b02a-497b-9a90-7f47a026e656\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") "
Nov 04 03:43:23.770268 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.769169    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c53728db-b02a-497b-9a90-7f47a026e656-kube-api-access-tbmgt\" (UniqueName: \"kubernetes.io/projected/c53728db-b02a-497b-9a90-7f47a026e656-kube-api-access-tbmgt\") pod \"c53728db-b02a-497b-9a90-7f47a026e656\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") "
Nov 04 03:43:23.770268 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.769270    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-tmp-dir\") pod \"c53728db-b02a-497b-9a90-7f47a026e656\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") "
Nov 04 03:43:23.770268 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.769335    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-dshm\") pod \"c53728db-b02a-497b-9a90-7f47a026e656\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") "
Nov 04 03:43:23.770268 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.769381    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c53728db-b02a-497b-9a90-7f47a026e656-tls-certs\" (UniqueName: \"kubernetes.io/secret/c53728db-b02a-497b-9a90-7f47a026e656-tls-certs\") pod \"c53728db-b02a-497b-9a90-7f47a026e656\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") "
Nov 04 03:43:23.770268 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.769421    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-model-cache\") pod \"c53728db-b02a-497b-9a90-7f47a026e656\" (UID: \"c53728db-b02a-497b-9a90-7f47a026e656\") "
Nov 04 03:43:23.771405 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.770630    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-tmp-dir" pod "c53728db-b02a-497b-9a90-7f47a026e656" (UID: "c53728db-b02a-497b-9a90-7f47a026e656"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:23.771405 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.770919    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-home" pod "c53728db-b02a-497b-9a90-7f47a026e656" (UID: "c53728db-b02a-497b-9a90-7f47a026e656"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:23.771862 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.771592    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-model-cache" pod "c53728db-b02a-497b-9a90-7f47a026e656" (UID: "c53728db-b02a-497b-9a90-7f47a026e656"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:23.775452 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.775417    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c53728db-b02a-497b-9a90-7f47a026e656-kube-api-access-tbmgt" pod "c53728db-b02a-497b-9a90-7f47a026e656" (UID: "c53728db-b02a-497b-9a90-7f47a026e656"). InnerVolumeSpecName "kube-api-access-tbmgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:23.776168 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.776137    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-dshm" pod "c53728db-b02a-497b-9a90-7f47a026e656" (UID: "c53728db-b02a-497b-9a90-7f47a026e656"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:23.809646 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.804088    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c53728db-b02a-497b-9a90-7f47a026e656-tls-certs" pod "c53728db-b02a-497b-9a90-7f47a026e656" (UID: "c53728db-b02a-497b-9a90-7f47a026e656"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:43:23.870756 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.870713    4521 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-home\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:23.870756 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.870753    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbmgt\" (UniqueName: \"kubernetes.io/projected/c53728db-b02a-497b-9a90-7f47a026e656-kube-api-access-tbmgt\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:23.870756 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.870763    4521 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-tmp-dir\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:23.871015 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.870773    4521 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-dshm\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:23.871015 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.870782    4521 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c53728db-b02a-497b-9a90-7f47a026e656-tls-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:23.871015 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:23.870791    4521 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c53728db-b02a-497b-9a90-7f47a026e656-model-cache\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:24.119462 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:24.119421    4521 generic.go:356] "Generic (PLEG): container finished" podID="c53728db-b02a-497b-9a90-7f47a026e656" containerID="dc8e5c1527037e5c8860e31cb626bf5fa1362517adfdaa0401f05b973f06a29b" exitCode=0
Nov 04 03:43:24.119710 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:24.119504    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"
Nov 04 03:43:24.119710 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:24.119628    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv" event={"ID":"c53728db-b02a-497b-9a90-7f47a026e656","Type":"ContainerDied","Data":"dc8e5c1527037e5c8860e31cb626bf5fa1362517adfdaa0401f05b973f06a29b"}
Nov 04 03:43:24.119710 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:24.119667    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv" event={"ID":"c53728db-b02a-497b-9a90-7f47a026e656","Type":"ContainerDied","Data":"8dc48279e6c2d296be0399b10618e7bb246b0e886102272b6705b5936137a0b3"}
Nov 04 03:43:24.119710 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:24.119695    4521 scope.go:122] "RemoveContainer" containerID="dc8e5c1527037e5c8860e31cb626bf5fa1362517adfdaa0401f05b973f06a29b"
Nov 04 03:43:24.131453 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:24.131363    4521 scope.go:122] "RemoveContainer" containerID="dc8e5c1527037e5c8860e31cb626bf5fa1362517adfdaa0401f05b973f06a29b"
Nov 04 03:43:24.131730 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:43:24.131700    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc8e5c1527037e5c8860e31cb626bf5fa1362517adfdaa0401f05b973f06a29b\": container with ID starting with dc8e5c1527037e5c8860e31cb626bf5fa1362517adfdaa0401f05b973f06a29b not found: ID does not exist" containerID="dc8e5c1527037e5c8860e31cb626bf5fa1362517adfdaa0401f05b973f06a29b"
Nov 04 03:43:24.131788 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:24.131735    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc8e5c1527037e5c8860e31cb626bf5fa1362517adfdaa0401f05b973f06a29b"} err="failed to get container status \"dc8e5c1527037e5c8860e31cb626bf5fa1362517adfdaa0401f05b973f06a29b\": rpc error: code = NotFound desc = could not find container \"dc8e5c1527037e5c8860e31cb626bf5fa1362517adfdaa0401f05b973f06a29b\": container with ID starting with dc8e5c1527037e5c8860e31cb626bf5fa1362517adfdaa0401f05b973f06a29b not found: ID does not exist"
Nov 04 03:43:24.288142 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:24.288107    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"]
Nov 04 03:43:24.338273 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:24.338237    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-82c724/premium-w3-82c724-kserve-9c894d4fc-gf8jv"]
Nov 04 03:43:25.912389 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:25.912313    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c53728db-b02a-497b-9a90-7f47a026e656" path="/var/lib/kubelet/pods/c53728db-b02a-497b-9a90-7f47a026e656/volumes"
Nov 04 03:43:28.729959 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:43:28.729915    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:43:29.859302 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:29.859266    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"]
Nov 04 03:43:29.859776 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:29.859461    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p" podUID="3cad7a57-df1d-4c61-ad35-5c8ddbaecc00" containerName="maas-api" containerID="cri-o://38f3e4cd31cbc3e9b2d9be7c282f76abe88d7a7d550f8b911d2f48d7ee0ece4c" gracePeriod=30
Nov 04 03:43:30.118108 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.118045    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:43:30.153526 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.153489    4521 generic.go:356] "Generic (PLEG): container finished" podID="3cad7a57-df1d-4c61-ad35-5c8ddbaecc00" containerID="38f3e4cd31cbc3e9b2d9be7c282f76abe88d7a7d550f8b911d2f48d7ee0ece4c" exitCode=0
Nov 04 03:43:30.153679 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.153534    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"
Nov 04 03:43:30.153679 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.153543    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p" event={"ID":"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00","Type":"ContainerDied","Data":"38f3e4cd31cbc3e9b2d9be7c282f76abe88d7a7d550f8b911d2f48d7ee0ece4c"}
Nov 04 03:43:30.153679 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.153588    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p" event={"ID":"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00","Type":"ContainerDied","Data":"f58f29363ed9b5f0fb4ed283a2a9e5ba5956864d9adc982b34a29d80bffba989"}
Nov 04 03:43:30.153679 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.153611    4521 scope.go:122] "RemoveContainer" containerID="38f3e4cd31cbc3e9b2d9be7c282f76abe88d7a7d550f8b911d2f48d7ee0ece4c"
Nov 04 03:43:30.162727 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.162706    4521 scope.go:122] "RemoveContainer" containerID="38f3e4cd31cbc3e9b2d9be7c282f76abe88d7a7d550f8b911d2f48d7ee0ece4c"
Nov 04 03:43:30.163012 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:43:30.162990    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38f3e4cd31cbc3e9b2d9be7c282f76abe88d7a7d550f8b911d2f48d7ee0ece4c\": container with ID starting with 38f3e4cd31cbc3e9b2d9be7c282f76abe88d7a7d550f8b911d2f48d7ee0ece4c not found: ID does not exist" containerID="38f3e4cd31cbc3e9b2d9be7c282f76abe88d7a7d550f8b911d2f48d7ee0ece4c"
Nov 04 03:43:30.163075 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.163021    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38f3e4cd31cbc3e9b2d9be7c282f76abe88d7a7d550f8b911d2f48d7ee0ece4c"} err="failed to get container status \"38f3e4cd31cbc3e9b2d9be7c282f76abe88d7a7d550f8b911d2f48d7ee0ece4c\": rpc error: code = NotFound desc = could not find container \"38f3e4cd31cbc3e9b2d9be7c282f76abe88d7a7d550f8b911d2f48d7ee0ece4c\": container with ID starting with 38f3e4cd31cbc3e9b2d9be7c282f76abe88d7a7d550f8b911d2f48d7ee0ece4c not found: ID does not exist"
Nov 04 03:43:30.242869 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.241934    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-kube-api-access-2r5jl\" (UniqueName: \"kubernetes.io/projected/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-kube-api-access-2r5jl\") pod \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\" (UID: \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\") "
Nov 04 03:43:30.242869 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.242185    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-metrics-certs\") pod \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\" (UID: \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\") "
Nov 04 03:43:30.242869 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.242433    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-openshift-service-ca\") pod \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\" (UID: \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\") "
Nov 04 03:43:30.242869 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.242473    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-maas-api-tls\") pod \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\" (UID: \"3cad7a57-df1d-4c61-ad35-5c8ddbaecc00\") "
Nov 04 03:43:30.242869 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.242801    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-openshift-service-ca" pod "3cad7a57-df1d-4c61-ad35-5c8ddbaecc00" (UID: "3cad7a57-df1d-4c61-ad35-5c8ddbaecc00"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:43:30.243250 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.243048    4521 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-openshift-service-ca\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:30.256093 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.256033    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-kube-api-access-2r5jl" pod "3cad7a57-df1d-4c61-ad35-5c8ddbaecc00" (UID: "3cad7a57-df1d-4c61-ad35-5c8ddbaecc00"). InnerVolumeSpecName "kube-api-access-2r5jl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:30.259774 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.259704    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-metrics-certs" pod "3cad7a57-df1d-4c61-ad35-5c8ddbaecc00" (UID: "3cad7a57-df1d-4c61-ad35-5c8ddbaecc00"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:43:30.262393 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.262307    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-maas-api-tls" pod "3cad7a57-df1d-4c61-ad35-5c8ddbaecc00" (UID: "3cad7a57-df1d-4c61-ad35-5c8ddbaecc00"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:43:30.343758 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.343716    4521 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-maas-api-tls\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:30.343758 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.343750    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2r5jl\" (UniqueName: \"kubernetes.io/projected/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-kube-api-access-2r5jl\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:30.343758 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.343761    4521 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00-metrics-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:30.587918 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.587883    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"]
Nov 04 03:43:30.593818 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:30.593791    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-50429e-78644ff8fb-fnx9p"]
Nov 04 03:43:31.911556 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:31.911522    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3cad7a57-df1d-4c61-ad35-5c8ddbaecc00" path="/var/lib/kubelet/pods/3cad7a57-df1d-4c61-ad35-5c8ddbaecc00/volumes"
Nov 04 03:43:46.585471 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:46.585427    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"]
Nov 04 03:43:46.585911 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:46.585662    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh" podUID="78ed61dd-e0d5-4165-ad93-58d761dfdd00" containerName="istio-proxy" containerID="cri-o://099a4693d988840f36bb62fa3381a283559edbf745a5504a1f829b3a626cf51e" gracePeriod=30
Nov 04 03:43:49.157486 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:49.157449    4521 patch_prober.go:29] interesting pod/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:43:49.157995 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:49.157525    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh" podUID="78ed61dd-e0d5-4165-ad93-58d761dfdd00" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:43:51.859391 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.859324    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:43:51.968140 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.968106    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78ed61dd-e0d5-4165-ad93-58d761dfdd00-kube-api-access-d9qd7\" (UniqueName: \"kubernetes.io/projected/78ed61dd-e0d5-4165-ad93-58d761dfdd00-kube-api-access-d9qd7\") pod \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") "
Nov 04 03:43:51.968316 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.968154    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-workload-socket\") pod \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") "
Nov 04 03:43:51.968316 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.968222    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-token\" (UniqueName: \"kubernetes.io/projected/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-token\") pod \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") "
Nov 04 03:43:51.968316 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.968278    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-credential-socket\") pod \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") "
Nov 04 03:43:51.968495 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.968321    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-data\") pod \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") "
Nov 04 03:43:51.968495 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.968380    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-workload-certs\") pod \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") "
Nov 04 03:43:51.968495 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.968428    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-envoy\") pod \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") "
Nov 04 03:43:51.968660 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.968493    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-podinfo\") pod \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") "
Nov 04 03:43:51.968660 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.968498    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-workload-socket" pod "78ed61dd-e0d5-4165-ad93-58d761dfdd00" (UID: "78ed61dd-e0d5-4165-ad93-58d761dfdd00"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:51.968660 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.968628    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istiod-ca-cert\") pod \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\" (UID: \"78ed61dd-e0d5-4165-ad93-58d761dfdd00\") "
Nov 04 03:43:51.968824 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.968649    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-data" pod "78ed61dd-e0d5-4165-ad93-58d761dfdd00" (UID: "78ed61dd-e0d5-4165-ad93-58d761dfdd00"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:51.968824 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.968661    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-workload-certs" pod "78ed61dd-e0d5-4165-ad93-58d761dfdd00" (UID: "78ed61dd-e0d5-4165-ad93-58d761dfdd00"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:51.968824 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.968543    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-credential-socket" pod "78ed61dd-e0d5-4165-ad93-58d761dfdd00" (UID: "78ed61dd-e0d5-4165-ad93-58d761dfdd00"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:51.969073 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.969049    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istiod-ca-cert" pod "78ed61dd-e0d5-4165-ad93-58d761dfdd00" (UID: "78ed61dd-e0d5-4165-ad93-58d761dfdd00"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:43:51.969247 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.969212    4521 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istiod-ca-cert\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:51.969247 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.969239    4521 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-workload-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:51.969382 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.969253    4521 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-credential-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:51.969382 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.969263    4521 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-data\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:51.969382 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.969271    4521 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-workload-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:51.970573 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.970543    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-token" pod "78ed61dd-e0d5-4165-ad93-58d761dfdd00" (UID: "78ed61dd-e0d5-4165-ad93-58d761dfdd00"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:51.970693 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.970676    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78ed61dd-e0d5-4165-ad93-58d761dfdd00-kube-api-access-d9qd7" pod "78ed61dd-e0d5-4165-ad93-58d761dfdd00" (UID: "78ed61dd-e0d5-4165-ad93-58d761dfdd00"). InnerVolumeSpecName "kube-api-access-d9qd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:51.971018 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.970993    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-envoy" pod "78ed61dd-e0d5-4165-ad93-58d761dfdd00" (UID: "78ed61dd-e0d5-4165-ad93-58d761dfdd00"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:51.971086 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:51.971025    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-podinfo" pod "78ed61dd-e0d5-4165-ad93-58d761dfdd00" (UID: "78ed61dd-e0d5-4165-ad93-58d761dfdd00"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:43:52.070691 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.070652    4521 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-envoy\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:52.070691 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.070686    4521 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-podinfo\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:52.070691 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.070698    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9qd7\" (UniqueName: \"kubernetes.io/projected/78ed61dd-e0d5-4165-ad93-58d761dfdd00-kube-api-access-d9qd7\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:52.070980 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.070711    4521 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78ed61dd-e0d5-4165-ad93-58d761dfdd00-istio-token\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:52.092320 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.092283    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"]
Nov 04 03:43:52.092554 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.092528    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg" podUID="c4d61e92-8c91-4bea-9811-4535ae9f00ef" containerName="maas-api" containerID="cri-o://5ee64d88b2f534cdc8a134f7a5fa3561f969be46479c6e4cc834ee5486eed425" gracePeriod=30
Nov 04 03:43:52.327285 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.327253    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:43:52.364477 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.364443    4521 generic.go:356] "Generic (PLEG): container finished" podID="c4d61e92-8c91-4bea-9811-4535ae9f00ef" containerID="5ee64d88b2f534cdc8a134f7a5fa3561f969be46479c6e4cc834ee5486eed425" exitCode=0
Nov 04 03:43:52.364636 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.364493    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"
Nov 04 03:43:52.364636 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.364519    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg" event={"ID":"c4d61e92-8c91-4bea-9811-4535ae9f00ef","Type":"ContainerDied","Data":"5ee64d88b2f534cdc8a134f7a5fa3561f969be46479c6e4cc834ee5486eed425"}
Nov 04 03:43:52.364636 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.364606    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg" event={"ID":"c4d61e92-8c91-4bea-9811-4535ae9f00ef","Type":"ContainerDied","Data":"7ad10456dd2389876e6650440df6b7c3e5b64a15d71974990a6968fe12b76077"}
Nov 04 03:43:52.364636 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.364633    4521 scope.go:122] "RemoveContainer" containerID="5ee64d88b2f534cdc8a134f7a5fa3561f969be46479c6e4cc834ee5486eed425"
Nov 04 03:43:52.366326 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.366303    4521 generic.go:356] "Generic (PLEG): container finished" podID="78ed61dd-e0d5-4165-ad93-58d761dfdd00" containerID="099a4693d988840f36bb62fa3381a283559edbf745a5504a1f829b3a626cf51e" exitCode=0
Nov 04 03:43:52.366455 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.366343    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh" event={"ID":"78ed61dd-e0d5-4165-ad93-58d761dfdd00","Type":"ContainerDied","Data":"099a4693d988840f36bb62fa3381a283559edbf745a5504a1f829b3a626cf51e"}
Nov 04 03:43:52.366455 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.366384    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh" event={"ID":"78ed61dd-e0d5-4165-ad93-58d761dfdd00","Type":"ContainerDied","Data":"61a1e77260be0ae7f1103f6799e9f8ddff6d55781d3ebabab2df28c15fe37aca"}
Nov 04 03:43:52.366455 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.366408    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"
Nov 04 03:43:52.374142 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.374127    4521 scope.go:122] "RemoveContainer" containerID="5ee64d88b2f534cdc8a134f7a5fa3561f969be46479c6e4cc834ee5486eed425"
Nov 04 03:43:52.374573 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:43:52.374442    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ee64d88b2f534cdc8a134f7a5fa3561f969be46479c6e4cc834ee5486eed425\": container with ID starting with 5ee64d88b2f534cdc8a134f7a5fa3561f969be46479c6e4cc834ee5486eed425 not found: ID does not exist" containerID="5ee64d88b2f534cdc8a134f7a5fa3561f969be46479c6e4cc834ee5486eed425"
Nov 04 03:43:52.374573 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.374476    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ee64d88b2f534cdc8a134f7a5fa3561f969be46479c6e4cc834ee5486eed425"} err="failed to get container status \"5ee64d88b2f534cdc8a134f7a5fa3561f969be46479c6e4cc834ee5486eed425\": rpc error: code = NotFound desc = could not find container \"5ee64d88b2f534cdc8a134f7a5fa3561f969be46479c6e4cc834ee5486eed425\": container with ID starting with 5ee64d88b2f534cdc8a134f7a5fa3561f969be46479c6e4cc834ee5486eed425 not found: ID does not exist"
Nov 04 03:43:52.374573 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.374510    4521 scope.go:122] "RemoveContainer" containerID="099a4693d988840f36bb62fa3381a283559edbf745a5504a1f829b3a626cf51e"
Nov 04 03:43:52.383663 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.383641    4521 scope.go:122] "RemoveContainer" containerID="82fe11f0362c0dd8c7e74ceeff0317b3ab313d309d1a831666bd0021f758de0f"
Nov 04 03:43:52.391874 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.391857    4521 scope.go:122] "RemoveContainer" containerID="099a4693d988840f36bb62fa3381a283559edbf745a5504a1f829b3a626cf51e"
Nov 04 03:43:52.392180 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:43:52.392160    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"099a4693d988840f36bb62fa3381a283559edbf745a5504a1f829b3a626cf51e\": container with ID starting with 099a4693d988840f36bb62fa3381a283559edbf745a5504a1f829b3a626cf51e not found: ID does not exist" containerID="099a4693d988840f36bb62fa3381a283559edbf745a5504a1f829b3a626cf51e"
Nov 04 03:43:52.392239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.392191    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"099a4693d988840f36bb62fa3381a283559edbf745a5504a1f829b3a626cf51e"} err="failed to get container status \"099a4693d988840f36bb62fa3381a283559edbf745a5504a1f829b3a626cf51e\": rpc error: code = NotFound desc = could not find container \"099a4693d988840f36bb62fa3381a283559edbf745a5504a1f829b3a626cf51e\": container with ID starting with 099a4693d988840f36bb62fa3381a283559edbf745a5504a1f829b3a626cf51e not found: ID does not exist"
Nov 04 03:43:52.392239 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.392209    4521 scope.go:122] "RemoveContainer" containerID="82fe11f0362c0dd8c7e74ceeff0317b3ab313d309d1a831666bd0021f758de0f"
Nov 04 03:43:52.392463 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:43:52.392445    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82fe11f0362c0dd8c7e74ceeff0317b3ab313d309d1a831666bd0021f758de0f\": container with ID starting with 82fe11f0362c0dd8c7e74ceeff0317b3ab313d309d1a831666bd0021f758de0f not found: ID does not exist" containerID="82fe11f0362c0dd8c7e74ceeff0317b3ab313d309d1a831666bd0021f758de0f"
Nov 04 03:43:52.392506 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.392468    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82fe11f0362c0dd8c7e74ceeff0317b3ab313d309d1a831666bd0021f758de0f"} err="failed to get container status \"82fe11f0362c0dd8c7e74ceeff0317b3ab313d309d1a831666bd0021f758de0f\": rpc error: code = NotFound desc = could not find container \"82fe11f0362c0dd8c7e74ceeff0317b3ab313d309d1a831666bd0021f758de0f\": container with ID starting with 82fe11f0362c0dd8c7e74ceeff0317b3ab313d309d1a831666bd0021f758de0f not found: ID does not exist"
Nov 04 03:43:52.475799 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.475629    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4d61e92-8c91-4bea-9811-4535ae9f00ef-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d61e92-8c91-4bea-9811-4535ae9f00ef-openshift-service-ca\") pod \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\" (UID: \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\") "
Nov 04 03:43:52.475799 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.475675    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c4d61e92-8c91-4bea-9811-4535ae9f00ef-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4d61e92-8c91-4bea-9811-4535ae9f00ef-metrics-certs\") pod \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\" (UID: \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\") "
Nov 04 03:43:52.475799 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.475714    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c4d61e92-8c91-4bea-9811-4535ae9f00ef-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4d61e92-8c91-4bea-9811-4535ae9f00ef-maas-api-tls\") pod \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\" (UID: \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\") "
Nov 04 03:43:52.476106 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.475902    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4d61e92-8c91-4bea-9811-4535ae9f00ef-kube-api-access-hdnzt\" (UniqueName: \"kubernetes.io/projected/c4d61e92-8c91-4bea-9811-4535ae9f00ef-kube-api-access-hdnzt\") pod \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\" (UID: \"c4d61e92-8c91-4bea-9811-4535ae9f00ef\") "
Nov 04 03:43:52.476166 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.476110    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4d61e92-8c91-4bea-9811-4535ae9f00ef-openshift-service-ca" pod "c4d61e92-8c91-4bea-9811-4535ae9f00ef" (UID: "c4d61e92-8c91-4bea-9811-4535ae9f00ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:43:52.476414 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.476384    4521 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d61e92-8c91-4bea-9811-4535ae9f00ef-openshift-service-ca\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:52.479017 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.478991    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4d61e92-8c91-4bea-9811-4535ae9f00ef-metrics-certs" pod "c4d61e92-8c91-4bea-9811-4535ae9f00ef" (UID: "c4d61e92-8c91-4bea-9811-4535ae9f00ef"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:43:52.479249 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.479226    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4d61e92-8c91-4bea-9811-4535ae9f00ef-kube-api-access-hdnzt" pod "c4d61e92-8c91-4bea-9811-4535ae9f00ef" (UID: "c4d61e92-8c91-4bea-9811-4535ae9f00ef"). InnerVolumeSpecName "kube-api-access-hdnzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:52.479386 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.479370    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4d61e92-8c91-4bea-9811-4535ae9f00ef-maas-api-tls" pod "c4d61e92-8c91-4bea-9811-4535ae9f00ef" (UID: "c4d61e92-8c91-4bea-9811-4535ae9f00ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:43:52.574304 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.574266    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"]
Nov 04 03:43:52.577676 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.577646    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdnzt\" (UniqueName: \"kubernetes.io/projected/c4d61e92-8c91-4bea-9811-4535ae9f00ef-kube-api-access-hdnzt\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:52.577676 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.577677    4521 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4d61e92-8c91-4bea-9811-4535ae9f00ef-metrics-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:52.577884 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.577688    4521 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4d61e92-8c91-4bea-9811-4535ae9f00ef-maas-api-tls\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:52.578004 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.577979    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-h4tsh"]
Nov 04 03:43:52.786654 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.786620    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"]
Nov 04 03:43:52.790858 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:52.790822    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-82c724-5874b8cc96-q8lgg"]
Nov 04 03:43:53.915973 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:53.915927    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78ed61dd-e0d5-4165-ad93-58d761dfdd00" path="/var/lib/kubelet/pods/78ed61dd-e0d5-4165-ad93-58d761dfdd00/volumes"
Nov 04 03:43:53.916564 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:53.916542    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4d61e92-8c91-4bea-9811-4535ae9f00ef" path="/var/lib/kubelet/pods/c4d61e92-8c91-4bea-9811-4535ae9f00ef/volumes"
Nov 04 03:43:54.682196 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.682152    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"]
Nov 04 03:43:54.682986 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.682959    4521 cpu_manager.go:414] "containerMap: removing container" podUID="1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1" containerName="payload-processing"
Nov 04 03:43:54.683111 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.682987    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1" containerName="payload-processing"
Nov 04 03:43:54.683111 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683007    4521 cpu_manager.go:414] "containerMap: removing container" podUID="78ed61dd-e0d5-4165-ad93-58d761dfdd00" containerName="istio-proxy"
Nov 04 03:43:54.683111 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683015    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78ed61dd-e0d5-4165-ad93-58d761dfdd00" containerName="istio-proxy"
Nov 04 03:43:54.683111 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683039    4521 cpu_manager.go:414] "containerMap: removing container" podUID="c4d61e92-8c91-4bea-9811-4535ae9f00ef" containerName="maas-api"
Nov 04 03:43:54.683111 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683048    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4d61e92-8c91-4bea-9811-4535ae9f00ef" containerName="maas-api"
Nov 04 03:43:54.683111 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683073    4521 cpu_manager.go:414] "containerMap: removing container" podUID="c8e3aedf-25cb-4002-8fd3-70b762747e3e" containerName="payload-pre-processing"
Nov 04 03:43:54.683111 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683081    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8e3aedf-25cb-4002-8fd3-70b762747e3e" containerName="payload-pre-processing"
Nov 04 03:43:54.683111 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683098    4521 cpu_manager.go:414] "containerMap: removing container" podUID="95615e76-0669-492a-b8a5-9b48b7fb1b15" containerName="payload-processing"
Nov 04 03:43:54.683111 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683106    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95615e76-0669-492a-b8a5-9b48b7fb1b15" containerName="payload-processing"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683124    4521 cpu_manager.go:414] "containerMap: removing container" podUID="3cad7a57-df1d-4c61-ad35-5c8ddbaecc00" containerName="maas-api"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683133    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3cad7a57-df1d-4c61-ad35-5c8ddbaecc00" containerName="maas-api"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683150    4521 cpu_manager.go:414] "containerMap: removing container" podUID="672c902d-f63e-42e6-b1e2-407d6e148c04" containerName="main"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683158    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="672c902d-f63e-42e6-b1e2-407d6e148c04" containerName="main"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683184    4521 cpu_manager.go:414] "containerMap: removing container" podUID="3483c5f0-468c-4b53-8efb-77788eae77a8" containerName="payload-processing"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683191    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3483c5f0-468c-4b53-8efb-77788eae77a8" containerName="payload-processing"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683203    4521 cpu_manager.go:414] "containerMap: removing container" podUID="69b690e9-f65d-42c9-9ca3-aea2ec55f30a" containerName="main"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683210    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69b690e9-f65d-42c9-9ca3-aea2ec55f30a" containerName="main"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683222    4521 cpu_manager.go:414] "containerMap: removing container" podUID="abf30d8d-9a93-4298-9ee3-92887e638513" containerName="maas-api"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683229    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abf30d8d-9a93-4298-9ee3-92887e638513" containerName="maas-api"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683240    4521 cpu_manager.go:414] "containerMap: removing container" podUID="c53728db-b02a-497b-9a90-7f47a026e656" containerName="main"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683248    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c53728db-b02a-497b-9a90-7f47a026e656" containerName="main"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683380    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="672c902d-f63e-42e6-b1e2-407d6e148c04" containerName="main"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683397    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="3483c5f0-468c-4b53-8efb-77788eae77a8" containerName="payload-processing"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683406    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="69b690e9-f65d-42c9-9ca3-aea2ec55f30a" containerName="main"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683415    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="abf30d8d-9a93-4298-9ee3-92887e638513" containerName="maas-api"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683424    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="c53728db-b02a-497b-9a90-7f47a026e656" containerName="main"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683434    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c3b45fd-7af2-4168-bc73-2a8e46e8d1e1" containerName="payload-processing"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683445    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="78ed61dd-e0d5-4165-ad93-58d761dfdd00" containerName="istio-proxy"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683456    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="78ed61dd-e0d5-4165-ad93-58d761dfdd00" containerName="istio-proxy"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683465    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4d61e92-8c91-4bea-9811-4535ae9f00ef" containerName="maas-api"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683476    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8e3aedf-25cb-4002-8fd3-70b762747e3e" containerName="payload-pre-processing"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683485    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="95615e76-0669-492a-b8a5-9b48b7fb1b15" containerName="payload-processing"
Nov 04 03:43:54.683602 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.683495    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="3cad7a57-df1d-4c61-ad35-5c8ddbaecc00" containerName="maas-api"
Nov 04 03:43:54.703975 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.697927    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.779868 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.776379    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-82c724-openshift-default-dockercfg-42cx7\""
Nov 04 03:43:54.821542 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.821500    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-data\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.822671 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.822639    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-credential-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.822933 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.822913    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8sbq\" (UniqueName: \"kubernetes.io/projected/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-kube-api-access-t8sbq\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.823098 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.823080    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-envoy\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.823230 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.823212    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-podinfo\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.823333 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.823316    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-token\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.823469 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.823452    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-workload-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.823600 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.823583    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-workload-certs\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.831874 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.823789    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istiod-ca-cert\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.867877 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.867722    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"]
Nov 04 03:43:54.933925 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.932903    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-podinfo\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.933925 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.932965    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-token\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.933925 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.933032    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-workload-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.933925 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.933072    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-workload-certs\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.933925 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.933204    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istiod-ca-cert\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.933925 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.933277    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-data\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.933925 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.933338    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-credential-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.933925 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.933379    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8sbq\" (UniqueName: \"kubernetes.io/projected/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-kube-api-access-t8sbq\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.933925 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.933453    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-envoy\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.958870 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.957271    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-credential-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:54.958870 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.957553    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-workload-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:55.024959 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.959364    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-data\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:55.024959 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:54.966050    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istiod-ca-cert\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:55.025216 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:55.025095    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-podinfo\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:55.025560 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:55.025535    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-workload-certs\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:55.048090 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:55.048025    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-token\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:55.097889 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:55.089632    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8sbq\" (UniqueName: \"kubernetes.io/projected/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-kube-api-access-t8sbq\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:55.106874 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:55.098523    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-envoy\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:55.355867 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:55.355120    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:56.512679 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:56.512653    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"]
Nov 04 03:43:56.514875 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:43:56.514824    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa1ef00d_7947_4df9_809e_1d3ddb5a4249.slice/crio-35e53186e201d07d4205510c744718db50290b83d48183854cc23dbbc92a7398 WatchSource:0}: Error finding container 35e53186e201d07d4205510c744718db50290b83d48183854cc23dbbc92a7398: Status 404 returned error can't find the container with id 35e53186e201d07d4205510c744718db50290b83d48183854cc23dbbc92a7398
Nov 04 03:43:56.517320 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:56.517280    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:43:56.517427 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:56.517366    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:43:56.517427 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:56.517412    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:43:57.424917 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:57.424874    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n" event={"ID":"aa1ef00d-7947-4df9-809e-1d3ddb5a4249","Type":"ContainerStarted","Data":"e4575f71b1dec092aa1987c2c67b868c559c162bccc468a8be8b17688b8dea0a"}
Nov 04 03:43:57.425099 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:57.424940    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n" event={"ID":"aa1ef00d-7947-4df9-809e-1d3ddb5a4249","Type":"ContainerStarted","Data":"35e53186e201d07d4205510c744718db50290b83d48183854cc23dbbc92a7398"}
Nov 04 03:43:57.606865 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:57.606662    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n" podStartSLOduration=3.6066378119999998 podStartE2EDuration="3.606637812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:43:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:43:57.55222627 +0000 UTC m=+1603.756826584" watchObservedRunningTime="2026-11-04 03:43:57.606637812 +0000 UTC m=+1603.811238116"
Nov 04 03:43:58.355988 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:58.355953    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:43:58.357365 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:58.357342    4521 patch_prober.go:29] interesting pod/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 04 03:43:58.357471 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:58.357396    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n" podUID="aa1ef00d-7947-4df9-809e-1d3ddb5a4249" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 04 03:43:58.776885 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:43:58.775640    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:43:59.355653 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:59.355610    4521 patch_prober.go:29] interesting pod/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 04 03:43:59.355913 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:43:59.355675    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n" podUID="aa1ef00d-7947-4df9-809e-1d3ddb5a4249" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 04 03:44:00.355999 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:00.355963    4521 patch_prober.go:29] interesting pod/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 04 03:44:00.356374 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:00.356016    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n" podUID="aa1ef00d-7947-4df9-809e-1d3ddb5a4249" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 04 03:44:01.355694 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:01.355647    4521 patch_prober.go:29] interesting pod/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 04 03:44:01.355932 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:01.355711    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n" podUID="aa1ef00d-7947-4df9-809e-1d3ddb5a4249" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 04 03:44:01.927121 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:01.927093    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"]
Nov 04 03:44:01.927760 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:01.927710    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n" podUID="aa1ef00d-7947-4df9-809e-1d3ddb5a4249" containerName="istio-proxy" containerID="cri-o://e4575f71b1dec092aa1987c2c67b868c559c162bccc468a8be8b17688b8dea0a" gracePeriod=30
Nov 04 03:44:04.977252 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:04.977214    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"]
Nov 04 03:44:04.977692 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:04.977424    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr" podUID="a8901771-3083-4017-93f8-331b57fe7b94" containerName="istio-proxy" containerID="cri-o://abac4d131f1f1efab5864ffdcf6e7ed665c1fd898330b4ceb74eed347b23b949" gracePeriod=30
Nov 04 03:44:08.183335 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.183310    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:44:08.316059 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.316025    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-credential-socket\") pod \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") "
Nov 04 03:44:08.316261 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.316112    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-workload-certs\") pod \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") "
Nov 04 03:44:08.316261 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.316161    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-envoy\") pod \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") "
Nov 04 03:44:08.316261 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.316189    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-data\") pod \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") "
Nov 04 03:44:08.316261 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.316215    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-token\" (UniqueName: \"kubernetes.io/projected/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-token\") pod \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") "
Nov 04 03:44:08.316261 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.316258    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-podinfo\") pod \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") "
Nov 04 03:44:08.316532 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.316348    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-kube-api-access-t8sbq\" (UniqueName: \"kubernetes.io/projected/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-kube-api-access-t8sbq\") pod \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") "
Nov 04 03:44:08.316532 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.316357    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-credential-socket" pod "aa1ef00d-7947-4df9-809e-1d3ddb5a4249" (UID: "aa1ef00d-7947-4df9-809e-1d3ddb5a4249"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:08.316532 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.316422    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-workload-certs" pod "aa1ef00d-7947-4df9-809e-1d3ddb5a4249" (UID: "aa1ef00d-7947-4df9-809e-1d3ddb5a4249"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:08.316532 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.316441    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istiod-ca-cert\") pod \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") "
Nov 04 03:44:08.316532 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.316500    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-workload-socket\") pod \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\" (UID: \"aa1ef00d-7947-4df9-809e-1d3ddb5a4249\") "
Nov 04 03:44:08.316791 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.316626    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-data" pod "aa1ef00d-7947-4df9-809e-1d3ddb5a4249" (UID: "aa1ef00d-7947-4df9-809e-1d3ddb5a4249"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:08.316884 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.316835    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istiod-ca-cert" pod "aa1ef00d-7947-4df9-809e-1d3ddb5a4249" (UID: "aa1ef00d-7947-4df9-809e-1d3ddb5a4249"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:44:08.316942 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.316895    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-workload-socket" pod "aa1ef00d-7947-4df9-809e-1d3ddb5a4249" (UID: "aa1ef00d-7947-4df9-809e-1d3ddb5a4249"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:08.317060 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.317035    4521 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istiod-ca-cert\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:08.317060 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.317061    4521 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-workload-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:08.317270 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.317076    4521 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-credential-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:08.317270 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.317089    4521 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-workload-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:08.317270 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.317103    4521 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-data\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:08.318597 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.318570    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-podinfo" pod "aa1ef00d-7947-4df9-809e-1d3ddb5a4249" (UID: "aa1ef00d-7947-4df9-809e-1d3ddb5a4249"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:44:08.318597 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.318586    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-kube-api-access-t8sbq" pod "aa1ef00d-7947-4df9-809e-1d3ddb5a4249" (UID: "aa1ef00d-7947-4df9-809e-1d3ddb5a4249"). InnerVolumeSpecName "kube-api-access-t8sbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:08.318806 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.318792    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-token" pod "aa1ef00d-7947-4df9-809e-1d3ddb5a4249" (UID: "aa1ef00d-7947-4df9-809e-1d3ddb5a4249"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:08.318967 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.318954    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-envoy" pod "aa1ef00d-7947-4df9-809e-1d3ddb5a4249" (UID: "aa1ef00d-7947-4df9-809e-1d3ddb5a4249"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:08.418492 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.418450    4521 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-envoy\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:08.418492 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.418482    4521 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-token\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:08.418697 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.418503    4521 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-istio-podinfo\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:08.418697 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.418514    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8sbq\" (UniqueName: \"kubernetes.io/projected/aa1ef00d-7947-4df9-809e-1d3ddb5a4249-kube-api-access-t8sbq\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:08.484951 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.484910    4521 generic.go:356] "Generic (PLEG): container finished" podID="aa1ef00d-7947-4df9-809e-1d3ddb5a4249" containerID="e4575f71b1dec092aa1987c2c67b868c559c162bccc468a8be8b17688b8dea0a" exitCode=0
Nov 04 03:44:08.485107 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.484968    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"
Nov 04 03:44:08.485107 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.484996    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n" event={"ID":"aa1ef00d-7947-4df9-809e-1d3ddb5a4249","Type":"ContainerDied","Data":"e4575f71b1dec092aa1987c2c67b868c559c162bccc468a8be8b17688b8dea0a"}
Nov 04 03:44:08.485107 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.485070    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n" event={"ID":"aa1ef00d-7947-4df9-809e-1d3ddb5a4249","Type":"ContainerDied","Data":"35e53186e201d07d4205510c744718db50290b83d48183854cc23dbbc92a7398"}
Nov 04 03:44:08.485253 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.485108    4521 scope.go:122] "RemoveContainer" containerID="e4575f71b1dec092aa1987c2c67b868c559c162bccc468a8be8b17688b8dea0a"
Nov 04 03:44:08.495892 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.495874    4521 scope.go:122] "RemoveContainer" containerID="e4575f71b1dec092aa1987c2c67b868c559c162bccc468a8be8b17688b8dea0a"
Nov 04 03:44:08.496187 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:44:08.496170    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4575f71b1dec092aa1987c2c67b868c559c162bccc468a8be8b17688b8dea0a\": container with ID starting with e4575f71b1dec092aa1987c2c67b868c559c162bccc468a8be8b17688b8dea0a not found: ID does not exist" containerID="e4575f71b1dec092aa1987c2c67b868c559c162bccc468a8be8b17688b8dea0a"
Nov 04 03:44:08.496228 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.496195    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4575f71b1dec092aa1987c2c67b868c559c162bccc468a8be8b17688b8dea0a"} err="failed to get container status \"e4575f71b1dec092aa1987c2c67b868c559c162bccc468a8be8b17688b8dea0a\": rpc error: code = NotFound desc = could not find container \"e4575f71b1dec092aa1987c2c67b868c559c162bccc468a8be8b17688b8dea0a\": container with ID starting with e4575f71b1dec092aa1987c2c67b868c559c162bccc468a8be8b17688b8dea0a not found: ID does not exist"
Nov 04 03:44:08.655616 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.655373    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"]
Nov 04 03:44:08.658203 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:08.658177    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-5vs5n"]
Nov 04 03:44:09.943060 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:09.943018    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa1ef00d-7947-4df9-809e-1d3ddb5a4249" path="/var/lib/kubelet/pods/aa1ef00d-7947-4df9-809e-1d3ddb5a4249/volumes"
Nov 04 03:44:10.667874 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:10.663671    4521 generic.go:356] "Generic (PLEG): container finished" podID="a8901771-3083-4017-93f8-331b57fe7b94" containerID="abac4d131f1f1efab5864ffdcf6e7ed665c1fd898330b4ceb74eed347b23b949" exitCode=0
Nov 04 03:44:10.668308 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:10.668206    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr" event={"ID":"a8901771-3083-4017-93f8-331b57fe7b94","Type":"ContainerDied","Data":"abac4d131f1f1efab5864ffdcf6e7ed665c1fd898330b4ceb74eed347b23b949"}
Nov 04 03:44:10.936878 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:10.933104    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:44:11.069969 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.062373    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-workload-certs\") pod \"a8901771-3083-4017-93f8-331b57fe7b94\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") "
Nov 04 03:44:11.069969 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.062460    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a8901771-3083-4017-93f8-331b57fe7b94-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8901771-3083-4017-93f8-331b57fe7b94-istio-podinfo\") pod \"a8901771-3083-4017-93f8-331b57fe7b94\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") "
Nov 04 03:44:11.069969 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.062516    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-credential-socket\") pod \"a8901771-3083-4017-93f8-331b57fe7b94\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") "
Nov 04 03:44:11.069969 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.062550    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-workload-socket\") pod \"a8901771-3083-4017-93f8-331b57fe7b94\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") "
Nov 04 03:44:11.069969 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.062581    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8901771-3083-4017-93f8-331b57fe7b94-kube-api-access-pr4h9\" (UniqueName: \"kubernetes.io/projected/a8901771-3083-4017-93f8-331b57fe7b94-kube-api-access-pr4h9\") pod \"a8901771-3083-4017-93f8-331b57fe7b94\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") "
Nov 04 03:44:11.069969 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.062651    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-istio-envoy\") pod \"a8901771-3083-4017-93f8-331b57fe7b94\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") "
Nov 04 03:44:11.069969 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.062736    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8901771-3083-4017-93f8-331b57fe7b94-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8901771-3083-4017-93f8-331b57fe7b94-istiod-ca-cert\") pod \"a8901771-3083-4017-93f8-331b57fe7b94\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") "
Nov 04 03:44:11.069969 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.062801    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8901771-3083-4017-93f8-331b57fe7b94-istio-token\" (UniqueName: \"kubernetes.io/projected/a8901771-3083-4017-93f8-331b57fe7b94-istio-token\") pod \"a8901771-3083-4017-93f8-331b57fe7b94\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") "
Nov 04 03:44:11.069969 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.062898    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-istio-data\") pod \"a8901771-3083-4017-93f8-331b57fe7b94\" (UID: \"a8901771-3083-4017-93f8-331b57fe7b94\") "
Nov 04 03:44:11.073293 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.073107    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-credential-socket" pod "a8901771-3083-4017-93f8-331b57fe7b94" (UID: "a8901771-3083-4017-93f8-331b57fe7b94"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:11.073703 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.073475    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-workload-socket" pod "a8901771-3083-4017-93f8-331b57fe7b94" (UID: "a8901771-3083-4017-93f8-331b57fe7b94"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:11.074024 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.074003    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-workload-certs" pod "a8901771-3083-4017-93f8-331b57fe7b94" (UID: "a8901771-3083-4017-93f8-331b57fe7b94"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:11.074615 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.074594    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-istio-data" pod "a8901771-3083-4017-93f8-331b57fe7b94" (UID: "a8901771-3083-4017-93f8-331b57fe7b94"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:11.075040 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.075020    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8901771-3083-4017-93f8-331b57fe7b94-istiod-ca-cert" pod "a8901771-3083-4017-93f8-331b57fe7b94" (UID: "a8901771-3083-4017-93f8-331b57fe7b94"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:44:11.079954 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.079931    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a8901771-3083-4017-93f8-331b57fe7b94-istio-podinfo" pod "a8901771-3083-4017-93f8-331b57fe7b94" (UID: "a8901771-3083-4017-93f8-331b57fe7b94"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:44:11.135871 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.112420    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8901771-3083-4017-93f8-331b57fe7b94-kube-api-access-pr4h9" pod "a8901771-3083-4017-93f8-331b57fe7b94" (UID: "a8901771-3083-4017-93f8-331b57fe7b94"). InnerVolumeSpecName "kube-api-access-pr4h9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:11.141866 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.141066    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8901771-3083-4017-93f8-331b57fe7b94-istio-token" pod "a8901771-3083-4017-93f8-331b57fe7b94" (UID: "a8901771-3083-4017-93f8-331b57fe7b94"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:11.153091 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.151989    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-istio-envoy" pod "a8901771-3083-4017-93f8-331b57fe7b94" (UID: "a8901771-3083-4017-93f8-331b57fe7b94"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:11.172070 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.172032    4521 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-workload-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:11.197867 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.176191    4521 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8901771-3083-4017-93f8-331b57fe7b94-istio-podinfo\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:11.293915 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.200904    4521 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-credential-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:11.293915 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.200956    4521 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-workload-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:11.293915 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.200972    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pr4h9\" (UniqueName: \"kubernetes.io/projected/a8901771-3083-4017-93f8-331b57fe7b94-kube-api-access-pr4h9\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:11.293915 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.200987    4521 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-istio-envoy\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:11.293915 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.201002    4521 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8901771-3083-4017-93f8-331b57fe7b94-istiod-ca-cert\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:11.293915 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.201031    4521 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8901771-3083-4017-93f8-331b57fe7b94-istio-token\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:11.293915 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.201045    4521 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8901771-3083-4017-93f8-331b57fe7b94-istio-data\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:11.657475 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.657429    4521 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:44:11.657810 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.657775    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr" podUID="a8901771-3083-4017-93f8-331b57fe7b94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:44:11.684535 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.684497    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr" event={"ID":"a8901771-3083-4017-93f8-331b57fe7b94","Type":"ContainerDied","Data":"928aded69819986bb75970f7d2ca9ed0a3306301a52a3fb849eaf80e7ace6f6e"}
Nov 04 03:44:11.684777 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.684759    4521 scope.go:122] "RemoveContainer" containerID="abac4d131f1f1efab5864ffdcf6e7ed665c1fd898330b4ceb74eed347b23b949"
Nov 04 03:44:11.689082 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.684548    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"
Nov 04 03:44:11.889013 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.888978    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"]
Nov 04 03:44:11.935920 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:11.927180    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-z97xr"]
Nov 04 03:44:13.912030 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:13.912001    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8901771-3083-4017-93f8-331b57fe7b94" path="/var/lib/kubelet/pods/a8901771-3083-4017-93f8-331b57fe7b94/volumes"
Nov 04 03:44:28.730931 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:44:28.730889    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:44:34.529363 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.529328    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"]
Nov 04 03:44:34.529806 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.529558    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2" podUID="82634af9-adbd-4085-9f4b-a4ab21d7b2d3" containerName="maas-api" containerID="cri-o://3e7ebea143df986c9771b01444d1e79b355891827b6b48a8a6bb9ab430084bf5" gracePeriod=30
Nov 04 03:44:34.625343 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.625301    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2" podUID="82634af9-adbd-4085-9f4b-a4ab21d7b2d3" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.53:8443/health\": dial tcp 10.134.0.53:8443: connect: connection refused"
Nov 04 03:44:34.795277 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.795209    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:44:34.810540 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.810510    4521 generic.go:356] "Generic (PLEG): container finished" podID="82634af9-adbd-4085-9f4b-a4ab21d7b2d3" containerID="3e7ebea143df986c9771b01444d1e79b355891827b6b48a8a6bb9ab430084bf5" exitCode=0
Nov 04 03:44:34.810703 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.810572    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"
Nov 04 03:44:34.810703 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.810603    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2" event={"ID":"82634af9-adbd-4085-9f4b-a4ab21d7b2d3","Type":"ContainerDied","Data":"3e7ebea143df986c9771b01444d1e79b355891827b6b48a8a6bb9ab430084bf5"}
Nov 04 03:44:34.810703 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.810670    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2" event={"ID":"82634af9-adbd-4085-9f4b-a4ab21d7b2d3","Type":"ContainerDied","Data":"a5612fdf5c296804714db08d4d2cbc1fa259f7978688dfa9e8dca061da05836e"}
Nov 04 03:44:34.810703 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.810701    4521 scope.go:122] "RemoveContainer" containerID="3e7ebea143df986c9771b01444d1e79b355891827b6b48a8a6bb9ab430084bf5"
Nov 04 03:44:34.820118 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.820099    4521 scope.go:122] "RemoveContainer" containerID="3e7ebea143df986c9771b01444d1e79b355891827b6b48a8a6bb9ab430084bf5"
Nov 04 03:44:34.820371 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:44:34.820353    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e7ebea143df986c9771b01444d1e79b355891827b6b48a8a6bb9ab430084bf5\": container with ID starting with 3e7ebea143df986c9771b01444d1e79b355891827b6b48a8a6bb9ab430084bf5 not found: ID does not exist" containerID="3e7ebea143df986c9771b01444d1e79b355891827b6b48a8a6bb9ab430084bf5"
Nov 04 03:44:34.820415 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.820379    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e7ebea143df986c9771b01444d1e79b355891827b6b48a8a6bb9ab430084bf5"} err="failed to get container status \"3e7ebea143df986c9771b01444d1e79b355891827b6b48a8a6bb9ab430084bf5\": rpc error: code = NotFound desc = could not find container \"3e7ebea143df986c9771b01444d1e79b355891827b6b48a8a6bb9ab430084bf5\": container with ID starting with 3e7ebea143df986c9771b01444d1e79b355891827b6b48a8a6bb9ab430084bf5 not found: ID does not exist"
Nov 04 03:44:34.851764 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.851731    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-openshift-service-ca\") pod \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\" (UID: \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\") "
Nov 04 03:44:34.851918 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.851889    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-metrics-certs\") pod \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\" (UID: \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\") "
Nov 04 03:44:34.851975 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.851936    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-kube-api-access-9mfbf\" (UniqueName: \"kubernetes.io/projected/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-kube-api-access-9mfbf\") pod \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\" (UID: \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\") "
Nov 04 03:44:34.852044 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.852016    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-maas-api-tls\") pod \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\" (UID: \"82634af9-adbd-4085-9f4b-a4ab21d7b2d3\") "
Nov 04 03:44:34.852108 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.852091    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-openshift-service-ca" pod "82634af9-adbd-4085-9f4b-a4ab21d7b2d3" (UID: "82634af9-adbd-4085-9f4b-a4ab21d7b2d3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:44:34.852517 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.852496    4521 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-openshift-service-ca\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:34.853937 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.853913    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-metrics-certs" pod "82634af9-adbd-4085-9f4b-a4ab21d7b2d3" (UID: "82634af9-adbd-4085-9f4b-a4ab21d7b2d3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:44:34.854053 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.854036    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-kube-api-access-9mfbf" pod "82634af9-adbd-4085-9f4b-a4ab21d7b2d3" (UID: "82634af9-adbd-4085-9f4b-a4ab21d7b2d3"). InnerVolumeSpecName "kube-api-access-9mfbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:34.854130 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.854112    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-maas-api-tls" pod "82634af9-adbd-4085-9f4b-a4ab21d7b2d3" (UID: "82634af9-adbd-4085-9f4b-a4ab21d7b2d3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:44:34.953995 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.953959    4521 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-metrics-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:34.953995 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.953997    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mfbf\" (UniqueName: \"kubernetes.io/projected/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-kube-api-access-9mfbf\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:34.954265 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:34.954012    4521 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82634af9-adbd-4085-9f4b-a4ab21d7b2d3-maas-api-tls\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:35.238858 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.238815    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"]
Nov 04 03:44:35.240739 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.240716    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"]
Nov 04 03:44:35.241194 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.241181    4521 cpu_manager.go:414] "containerMap: removing container" podUID="aa1ef00d-7947-4df9-809e-1d3ddb5a4249" containerName="istio-proxy"
Nov 04 03:44:35.241234 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.241196    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa1ef00d-7947-4df9-809e-1d3ddb5a4249" containerName="istio-proxy"
Nov 04 03:44:35.241234 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.241218    4521 cpu_manager.go:414] "containerMap: removing container" podUID="82634af9-adbd-4085-9f4b-a4ab21d7b2d3" containerName="maas-api"
Nov 04 03:44:35.241234 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.241223    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82634af9-adbd-4085-9f4b-a4ab21d7b2d3" containerName="maas-api"
Nov 04 03:44:35.241234 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.241232    4521 cpu_manager.go:414] "containerMap: removing container" podUID="78ed61dd-e0d5-4165-ad93-58d761dfdd00" containerName="istio-proxy"
Nov 04 03:44:35.241357 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.241237    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78ed61dd-e0d5-4165-ad93-58d761dfdd00" containerName="istio-proxy"
Nov 04 03:44:35.241357 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.241247    4521 cpu_manager.go:414] "containerMap: removing container" podUID="a8901771-3083-4017-93f8-331b57fe7b94" containerName="istio-proxy"
Nov 04 03:44:35.241357 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.241252    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8901771-3083-4017-93f8-331b57fe7b94" containerName="istio-proxy"
Nov 04 03:44:35.241357 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.241328    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa1ef00d-7947-4df9-809e-1d3ddb5a4249" containerName="istio-proxy"
Nov 04 03:44:35.241357 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.241339    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="82634af9-adbd-4085-9f4b-a4ab21d7b2d3" containerName="maas-api"
Nov 04 03:44:35.241357 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.241345    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8901771-3083-4017-93f8-331b57fe7b94" containerName="istio-proxy"
Nov 04 03:44:35.244559 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.244541    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.247401 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.247380    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-092f8c-747cc679f5-xb6v2"]
Nov 04 03:44:35.300031 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.300002    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-092f8c-openshift-default-dockercfg-9bd4m\""
Nov 04 03:44:35.348694 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.348659    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"]
Nov 04 03:44:35.359096 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.359062    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.359279 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.359121    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.359279 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.359227    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4814588-6d91-4621-b701-065525865091-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.359279 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.359271    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.359403 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.359375    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.359448 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.359409    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4814588-6d91-4621-b701-065525865091-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.359544 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.359525    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.359590 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.359575    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djtdr\" (UniqueName: \"kubernetes.io/projected/d4814588-6d91-4621-b701-065525865091-kube-api-access-djtdr\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.359712 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.359701    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4814588-6d91-4621-b701-065525865091-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.461487 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.461407    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.461487 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.461471    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djtdr\" (UniqueName: \"kubernetes.io/projected/d4814588-6d91-4621-b701-065525865091-kube-api-access-djtdr\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.461715 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.461570    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4814588-6d91-4621-b701-065525865091-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.461715 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.461606    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.461715 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.461640    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.461715 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.461673    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4814588-6d91-4621-b701-065525865091-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.461715 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.461710    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.462009 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.461748    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.462009 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.461774    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4814588-6d91-4621-b701-065525865091-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.462576 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.462517    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.462576 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.462561    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.462980 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.462954    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.463268 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.463243    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.463549 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.463526    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4814588-6d91-4621-b701-065525865091-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.464188 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.464163    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.464477 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.464454    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4814588-6d91-4621-b701-065525865091-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.524013 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.523982    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4814588-6d91-4621-b701-065525865091-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.524013 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.523997    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djtdr\" (UniqueName: \"kubernetes.io/projected/d4814588-6d91-4621-b701-065525865091-kube-api-access-djtdr\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs\" (UID: \"d4814588-6d91-4621-b701-065525865091\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.557821 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.557784    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:35.911596 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.911550    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82634af9-adbd-4085-9f4b-a4ab21d7b2d3" path="/var/lib/kubelet/pods/82634af9-adbd-4085-9f4b-a4ab21d7b2d3/volumes"
Nov 04 03:44:35.987713 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.987685    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"]
Nov 04 03:44:35.989279 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:44:35.989249    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4814588_6d91_4621_b701_065525865091.slice/crio-6e22b015f5afe39c531a675e08c97cdbf7cf797f0d7b21f4831b97b44d961088 WatchSource:0}: Error finding container 6e22b015f5afe39c531a675e08c97cdbf7cf797f0d7b21f4831b97b44d961088: Status 404 returned error can't find the container with id 6e22b015f5afe39c531a675e08c97cdbf7cf797f0d7b21f4831b97b44d961088
Nov 04 03:44:35.991760 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.991715    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:44:35.991872 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.991810    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:44:35.991925 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:35.991905    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:44:36.831327 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:36.831288    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs" event={"ID":"d4814588-6d91-4621-b701-065525865091","Type":"ContainerStarted","Data":"3de9d8557c80a06d0b00e29b8ec7edb4dadb981d58138fc389a7ff5da7ffd125"}
Nov 04 03:44:36.831774 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:36.831364    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs" event={"ID":"d4814588-6d91-4621-b701-065525865091","Type":"ContainerStarted","Data":"6e22b015f5afe39c531a675e08c97cdbf7cf797f0d7b21f4831b97b44d961088"}
Nov 04 03:44:36.954825 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:36.954763    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs" podStartSLOduration=1.9547458500000001 podStartE2EDuration="1.95474585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:44:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:44:36.952677693 +0000 UTC m=+1643.157277994" watchObservedRunningTime="2026-11-04 03:44:36.95474585 +0000 UTC m=+1643.159346151"
Nov 04 03:44:37.558796 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:37.558749    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:37.560128 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:37.560100    4521 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 03:44:37.560262 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:37.560149    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs" podUID="d4814588-6d91-4621-b701-065525865091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 03:44:38.559048 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:38.559009    4521 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 03:44:38.559445 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:38.559064    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs" podUID="d4814588-6d91-4621-b701-065525865091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 03:44:39.558706 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:39.558667    4521 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 03:44:39.558917 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:39.558722    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs" podUID="d4814588-6d91-4621-b701-065525865091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 03:44:40.558864 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:40.558799    4521 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 03:44:40.559269 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:40.558877    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs" podUID="d4814588-6d91-4621-b701-065525865091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 03:44:41.558480 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:41.558444    4521 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 03:44:41.558679 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:41.558517    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs" podUID="d4814588-6d91-4621-b701-065525865091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 03:44:42.558724 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:42.558676    4521 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 03:44:42.559165 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:42.558749    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs" podUID="d4814588-6d91-4621-b701-065525865091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 03:44:43.853387 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:43.853355    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:43.853902 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:43.853702    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:43.854961 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:43.854940    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:45.828141 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:45.828096    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"]
Nov 04 03:44:45.882986 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:45.882936    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs" podUID="d4814588-6d91-4621-b701-065525865091" containerName="istio-proxy" containerID="cri-o://3de9d8557c80a06d0b00e29b8ec7edb4dadb981d58138fc389a7ff5da7ffd125" gracePeriod=30
Nov 04 03:44:51.149863 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.149792    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:51.277835 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.277806    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4814588-6d91-4621-b701-065525865091-kube-api-access-djtdr\" (UniqueName: \"kubernetes.io/projected/d4814588-6d91-4621-b701-065525865091-kube-api-access-djtdr\") pod \"d4814588-6d91-4621-b701-065525865091\" (UID: \"d4814588-6d91-4621-b701-065525865091\") "
Nov 04 03:44:51.277994 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.277881    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-istio-data\") pod \"d4814588-6d91-4621-b701-065525865091\" (UID: \"d4814588-6d91-4621-b701-065525865091\") "
Nov 04 03:44:51.277994 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.277909    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-credential-socket\") pod \"d4814588-6d91-4621-b701-065525865091\" (UID: \"d4814588-6d91-4621-b701-065525865091\") "
Nov 04 03:44:51.277994 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.277974    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-workload-socket\") pod \"d4814588-6d91-4621-b701-065525865091\" (UID: \"d4814588-6d91-4621-b701-065525865091\") "
Nov 04 03:44:51.278144 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.278004    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4814588-6d91-4621-b701-065525865091-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4814588-6d91-4621-b701-065525865091-istiod-ca-cert\") pod \"d4814588-6d91-4621-b701-065525865091\" (UID: \"d4814588-6d91-4621-b701-065525865091\") "
Nov 04 03:44:51.278144 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.278055    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4814588-6d91-4621-b701-065525865091-istio-token\" (UniqueName: \"kubernetes.io/projected/d4814588-6d91-4621-b701-065525865091-istio-token\") pod \"d4814588-6d91-4621-b701-065525865091\" (UID: \"d4814588-6d91-4621-b701-065525865091\") "
Nov 04 03:44:51.278144 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.278082    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d4814588-6d91-4621-b701-065525865091-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4814588-6d91-4621-b701-065525865091-istio-podinfo\") pod \"d4814588-6d91-4621-b701-065525865091\" (UID: \"d4814588-6d91-4621-b701-065525865091\") "
Nov 04 03:44:51.278325 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.278152    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-istio-envoy\") pod \"d4814588-6d91-4621-b701-065525865091\" (UID: \"d4814588-6d91-4621-b701-065525865091\") "
Nov 04 03:44:51.278325 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.278191    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-workload-certs\") pod \"d4814588-6d91-4621-b701-065525865091\" (UID: \"d4814588-6d91-4621-b701-065525865091\") "
Nov 04 03:44:51.278325 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.278213    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-istio-data" pod "d4814588-6d91-4621-b701-065525865091" (UID: "d4814588-6d91-4621-b701-065525865091"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.278486 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.278328    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-credential-socket" pod "d4814588-6d91-4621-b701-065525865091" (UID: "d4814588-6d91-4621-b701-065525865091"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.278486 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.278455    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4814588-6d91-4621-b701-065525865091-istiod-ca-cert" pod "d4814588-6d91-4621-b701-065525865091" (UID: "d4814588-6d91-4621-b701-065525865091"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:44:51.278486 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.278469    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-workload-socket" pod "d4814588-6d91-4621-b701-065525865091" (UID: "d4814588-6d91-4621-b701-065525865091"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.278636 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.278573    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-workload-certs" pod "d4814588-6d91-4621-b701-065525865091" (UID: "d4814588-6d91-4621-b701-065525865091"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.278760 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.278742    4521 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-istio-data\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.278824 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.278765    4521 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-credential-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.278824 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.278780    4521 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-workload-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.278824 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.278795    4521 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4814588-6d91-4621-b701-065525865091-istiod-ca-cert\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.278824 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.278809    4521 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-workload-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.280144 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.280114    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4814588-6d91-4621-b701-065525865091-kube-api-access-djtdr" pod "d4814588-6d91-4621-b701-065525865091" (UID: "d4814588-6d91-4621-b701-065525865091"). InnerVolumeSpecName "kube-api-access-djtdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:51.280249 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.280211    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-istio-envoy" pod "d4814588-6d91-4621-b701-065525865091" (UID: "d4814588-6d91-4621-b701-065525865091"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.280725 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.280702    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4814588-6d91-4621-b701-065525865091-istio-podinfo" pod "d4814588-6d91-4621-b701-065525865091" (UID: "d4814588-6d91-4621-b701-065525865091"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:44:51.280725 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.280716    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4814588-6d91-4621-b701-065525865091-istio-token" pod "d4814588-6d91-4621-b701-065525865091" (UID: "d4814588-6d91-4621-b701-065525865091"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:51.379725 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.379691    4521 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4814588-6d91-4621-b701-065525865091-istio-token\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.379725 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.379723    4521 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4814588-6d91-4621-b701-065525865091-istio-podinfo\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.379946 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.379734    4521 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4814588-6d91-4621-b701-065525865091-istio-envoy\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.379946 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.379744    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djtdr\" (UniqueName: \"kubernetes.io/projected/d4814588-6d91-4621-b701-065525865091-kube-api-access-djtdr\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.916948 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.916909    4521 generic.go:356] "Generic (PLEG): container finished" podID="d4814588-6d91-4621-b701-065525865091" containerID="3de9d8557c80a06d0b00e29b8ec7edb4dadb981d58138fc389a7ff5da7ffd125" exitCode=0
Nov 04 03:44:51.917131 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.916974    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"
Nov 04 03:44:51.917131 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.916995    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs" event={"ID":"d4814588-6d91-4621-b701-065525865091","Type":"ContainerDied","Data":"3de9d8557c80a06d0b00e29b8ec7edb4dadb981d58138fc389a7ff5da7ffd125"}
Nov 04 03:44:51.917131 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.917061    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs" event={"ID":"d4814588-6d91-4621-b701-065525865091","Type":"ContainerDied","Data":"6e22b015f5afe39c531a675e08c97cdbf7cf797f0d7b21f4831b97b44d961088"}
Nov 04 03:44:51.917131 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.917093    4521 scope.go:122] "RemoveContainer" containerID="3de9d8557c80a06d0b00e29b8ec7edb4dadb981d58138fc389a7ff5da7ffd125"
Nov 04 03:44:51.927241 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.927222    4521 scope.go:122] "RemoveContainer" containerID="3de9d8557c80a06d0b00e29b8ec7edb4dadb981d58138fc389a7ff5da7ffd125"
Nov 04 03:44:51.927520 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:44:51.927501    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3de9d8557c80a06d0b00e29b8ec7edb4dadb981d58138fc389a7ff5da7ffd125\": container with ID starting with 3de9d8557c80a06d0b00e29b8ec7edb4dadb981d58138fc389a7ff5da7ffd125 not found: ID does not exist" containerID="3de9d8557c80a06d0b00e29b8ec7edb4dadb981d58138fc389a7ff5da7ffd125"
Nov 04 03:44:51.927567 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:51.927529    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3de9d8557c80a06d0b00e29b8ec7edb4dadb981d58138fc389a7ff5da7ffd125"} err="failed to get container status \"3de9d8557c80a06d0b00e29b8ec7edb4dadb981d58138fc389a7ff5da7ffd125\": rpc error: code = NotFound desc = could not find container \"3de9d8557c80a06d0b00e29b8ec7edb4dadb981d58138fc389a7ff5da7ffd125\": container with ID starting with 3de9d8557c80a06d0b00e29b8ec7edb4dadb981d58138fc389a7ff5da7ffd125 not found: ID does not exist"
Nov 04 03:44:52.084261 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:52.084220    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"]
Nov 04 03:44:52.134406 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:52.134374    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-f5sjs"]
Nov 04 03:44:53.911710 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:44:53.911674    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4814588-6d91-4621-b701-065525865091" path="/var/lib/kubelet/pods/d4814588-6d91-4621-b701-065525865091/volumes"
Nov 04 03:44:58.728606 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:44:58.728524    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:45:28.729783 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:45:28.729743    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:45:58.728969 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:45:58.728933    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:46:28.729314 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:46:28.729229    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:46:58.729714 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:46:58.729671    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:47:28.729304 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:47:28.729265    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:47:58.729826 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:47:58.729740    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:48:28.729506 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:48:28.729457    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:48:58.729617 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:48:58.729572    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:49:28.729453 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:49:28.729362    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:49:29.669740 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:29.669702    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"]
Nov 04 03:49:29.670107 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:29.670069    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl" podUID="a9ea7e36-1384-4ab8-9f5c-a5562faba8c2" containerName="maas-api" containerID="cri-o://eb36dcb39a77e726e639ad1a2ab12a0eb7c28c93bbd796efc65efafcdc4cf063" gracePeriod=30
Nov 04 03:49:29.946068 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:29.946034    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:49:30.004374 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.004342    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-kube-api-access-2pd6j\" (UniqueName: \"kubernetes.io/projected/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-kube-api-access-2pd6j\") pod \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\" (UID: \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\") "
Nov 04 03:49:30.004567 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.004391    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-openshift-service-ca\") pod \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\" (UID: \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\") "
Nov 04 03:49:30.004567 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.004546    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-maas-api-tls\") pod \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\" (UID: \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\") "
Nov 04 03:49:30.004706 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.004575    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-metrics-certs\") pod \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\" (UID: \"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2\") "
Nov 04 03:49:30.004856 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.004821    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-openshift-service-ca" pod "a9ea7e36-1384-4ab8-9f5c-a5562faba8c2" (UID: "a9ea7e36-1384-4ab8-9f5c-a5562faba8c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:49:30.005361 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.005153    4521 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-openshift-service-ca\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:30.007057 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.006982    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-kube-api-access-2pd6j" pod "a9ea7e36-1384-4ab8-9f5c-a5562faba8c2" (UID: "a9ea7e36-1384-4ab8-9f5c-a5562faba8c2"). InnerVolumeSpecName "kube-api-access-2pd6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:49:30.007161 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.007087    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-maas-api-tls" pod "a9ea7e36-1384-4ab8-9f5c-a5562faba8c2" (UID: "a9ea7e36-1384-4ab8-9f5c-a5562faba8c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:49:30.007161 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.007111    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-metrics-certs" pod "a9ea7e36-1384-4ab8-9f5c-a5562faba8c2" (UID: "a9ea7e36-1384-4ab8-9f5c-a5562faba8c2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:49:30.106545 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.106498    4521 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-maas-api-tls\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:30.106698 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.106550    4521 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-metrics-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:30.106698 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.106568    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pd6j\" (UniqueName: \"kubernetes.io/projected/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2-kube-api-access-2pd6j\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:30.378034 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.377995    4521 generic.go:356] "Generic (PLEG): container finished" podID="a9ea7e36-1384-4ab8-9f5c-a5562faba8c2" containerID="eb36dcb39a77e726e639ad1a2ab12a0eb7c28c93bbd796efc65efafcdc4cf063" exitCode=0
Nov 04 03:49:30.378222 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.378042    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"
Nov 04 03:49:30.378222 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.378074    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl" event={"ID":"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2","Type":"ContainerDied","Data":"eb36dcb39a77e726e639ad1a2ab12a0eb7c28c93bbd796efc65efafcdc4cf063"}
Nov 04 03:49:30.378222 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.378124    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl" event={"ID":"a9ea7e36-1384-4ab8-9f5c-a5562faba8c2","Type":"ContainerDied","Data":"b0837bd5aba1f0877af0c0560bb9c0be3dbffa71ed95e2003f0c7d442689333d"}
Nov 04 03:49:30.378222 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.378145    4521 scope.go:122] "RemoveContainer" containerID="eb36dcb39a77e726e639ad1a2ab12a0eb7c28c93bbd796efc65efafcdc4cf063"
Nov 04 03:49:30.387948 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.387929    4521 scope.go:122] "RemoveContainer" containerID="eb36dcb39a77e726e639ad1a2ab12a0eb7c28c93bbd796efc65efafcdc4cf063"
Nov 04 03:49:30.388220 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:49:30.388200    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb36dcb39a77e726e639ad1a2ab12a0eb7c28c93bbd796efc65efafcdc4cf063\": container with ID starting with eb36dcb39a77e726e639ad1a2ab12a0eb7c28c93bbd796efc65efafcdc4cf063 not found: ID does not exist" containerID="eb36dcb39a77e726e639ad1a2ab12a0eb7c28c93bbd796efc65efafcdc4cf063"
Nov 04 03:49:30.388273 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.388227    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb36dcb39a77e726e639ad1a2ab12a0eb7c28c93bbd796efc65efafcdc4cf063"} err="failed to get container status \"eb36dcb39a77e726e639ad1a2ab12a0eb7c28c93bbd796efc65efafcdc4cf063\": rpc error: code = NotFound desc = could not find container \"eb36dcb39a77e726e639ad1a2ab12a0eb7c28c93bbd796efc65efafcdc4cf063\": container with ID starting with eb36dcb39a77e726e639ad1a2ab12a0eb7c28c93bbd796efc65efafcdc4cf063 not found: ID does not exist"
Nov 04 03:49:30.545066 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.545027    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"]
Nov 04 03:49:30.595555 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:30.595516    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6d170d-778685747d-6fbtl"]
Nov 04 03:49:31.911046 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:31.911011    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9ea7e36-1384-4ab8-9f5c-a5562faba8c2" path="/var/lib/kubelet/pods/a9ea7e36-1384-4ab8-9f5c-a5562faba8c2/volumes"
Nov 04 03:49:45.639924 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:45.639887    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"]
Nov 04 03:49:46.457896 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:46.457860    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"]
Nov 04 03:49:46.458155 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:46.458115    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s" podUID="53e6e076-af21-469c-9713-6b7f608833bc" containerName="maas-api" containerID="cri-o://8065a90ffcd9e50a11628dad6fb2f728872733d1053ce9f46064d69fea701830" gracePeriod=30
Nov 04 03:49:46.727660 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:46.727595    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:49:46.783169 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:46.783135    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/53e6e076-af21-469c-9713-6b7f608833bc-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53e6e076-af21-469c-9713-6b7f608833bc-maas-api-tls\") pod \"53e6e076-af21-469c-9713-6b7f608833bc\" (UID: \"53e6e076-af21-469c-9713-6b7f608833bc\") "
Nov 04 03:49:46.783347 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:46.783219    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/53e6e076-af21-469c-9713-6b7f608833bc-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53e6e076-af21-469c-9713-6b7f608833bc-openshift-service-ca\") pod \"53e6e076-af21-469c-9713-6b7f608833bc\" (UID: \"53e6e076-af21-469c-9713-6b7f608833bc\") "
Nov 04 03:49:46.783347 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:46.783291    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/53e6e076-af21-469c-9713-6b7f608833bc-metrics-certs\" (UniqueName: \"kubernetes.io/secret/53e6e076-af21-469c-9713-6b7f608833bc-metrics-certs\") pod \"53e6e076-af21-469c-9713-6b7f608833bc\" (UID: \"53e6e076-af21-469c-9713-6b7f608833bc\") "
Nov 04 03:49:46.783477 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:46.783356    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53e6e076-af21-469c-9713-6b7f608833bc-kube-api-access-lt8sm\" (UniqueName: \"kubernetes.io/projected/53e6e076-af21-469c-9713-6b7f608833bc-kube-api-access-lt8sm\") pod \"53e6e076-af21-469c-9713-6b7f608833bc\" (UID: \"53e6e076-af21-469c-9713-6b7f608833bc\") "
Nov 04 03:49:46.783655 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:46.783622    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53e6e076-af21-469c-9713-6b7f608833bc-openshift-service-ca" pod "53e6e076-af21-469c-9713-6b7f608833bc" (UID: "53e6e076-af21-469c-9713-6b7f608833bc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:49:46.785434 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:46.785401    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53e6e076-af21-469c-9713-6b7f608833bc-metrics-certs" pod "53e6e076-af21-469c-9713-6b7f608833bc" (UID: "53e6e076-af21-469c-9713-6b7f608833bc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:49:46.785434 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:46.785413    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53e6e076-af21-469c-9713-6b7f608833bc-kube-api-access-lt8sm" pod "53e6e076-af21-469c-9713-6b7f608833bc" (UID: "53e6e076-af21-469c-9713-6b7f608833bc"). InnerVolumeSpecName "kube-api-access-lt8sm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:49:46.785649 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:46.785632    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53e6e076-af21-469c-9713-6b7f608833bc-maas-api-tls" pod "53e6e076-af21-469c-9713-6b7f608833bc" (UID: "53e6e076-af21-469c-9713-6b7f608833bc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:49:46.885293 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:46.885254    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lt8sm\" (UniqueName: \"kubernetes.io/projected/53e6e076-af21-469c-9713-6b7f608833bc-kube-api-access-lt8sm\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:46.885293 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:46.885282    4521 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53e6e076-af21-469c-9713-6b7f608833bc-maas-api-tls\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:46.885293 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:46.885293    4521 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53e6e076-af21-469c-9713-6b7f608833bc-openshift-service-ca\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:46.885535 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:46.885308    4521 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53e6e076-af21-469c-9713-6b7f608833bc-metrics-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:47.467228 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:47.467194    4521 generic.go:356] "Generic (PLEG): container finished" podID="53e6e076-af21-469c-9713-6b7f608833bc" containerID="8065a90ffcd9e50a11628dad6fb2f728872733d1053ce9f46064d69fea701830" exitCode=0
Nov 04 03:49:47.467423 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:47.467239    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"
Nov 04 03:49:47.467423 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:47.467268    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s" event={"ID":"53e6e076-af21-469c-9713-6b7f608833bc","Type":"ContainerDied","Data":"8065a90ffcd9e50a11628dad6fb2f728872733d1053ce9f46064d69fea701830"}
Nov 04 03:49:47.467423 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:47.467308    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s" event={"ID":"53e6e076-af21-469c-9713-6b7f608833bc","Type":"ContainerDied","Data":"fad7dffb8eda685843ae5157bb19507c1e00af69f9a71b543c3728897be79d58"}
Nov 04 03:49:47.467423 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:47.467330    4521 scope.go:122] "RemoveContainer" containerID="8065a90ffcd9e50a11628dad6fb2f728872733d1053ce9f46064d69fea701830"
Nov 04 03:49:47.477735 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:47.477716    4521 scope.go:122] "RemoveContainer" containerID="8065a90ffcd9e50a11628dad6fb2f728872733d1053ce9f46064d69fea701830"
Nov 04 03:49:47.478041 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:49:47.478019    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8065a90ffcd9e50a11628dad6fb2f728872733d1053ce9f46064d69fea701830\": container with ID starting with 8065a90ffcd9e50a11628dad6fb2f728872733d1053ce9f46064d69fea701830 not found: ID does not exist" containerID="8065a90ffcd9e50a11628dad6fb2f728872733d1053ce9f46064d69fea701830"
Nov 04 03:49:47.478131 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:47.478049    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8065a90ffcd9e50a11628dad6fb2f728872733d1053ce9f46064d69fea701830"} err="failed to get container status \"8065a90ffcd9e50a11628dad6fb2f728872733d1053ce9f46064d69fea701830\": rpc error: code = NotFound desc = could not find container \"8065a90ffcd9e50a11628dad6fb2f728872733d1053ce9f46064d69fea701830\": container with ID starting with 8065a90ffcd9e50a11628dad6fb2f728872733d1053ce9f46064d69fea701830 not found: ID does not exist"
Nov 04 03:49:47.634252 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:47.634212    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"]
Nov 04 03:49:47.638609 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:47.638572    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-951893-6fcffd4664-ngv9s"]
Nov 04 03:49:47.911052 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:47.911018    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="53e6e076-af21-469c-9713-6b7f608833bc" path="/var/lib/kubelet/pods/53e6e076-af21-469c-9713-6b7f608833bc/volumes"
Nov 04 03:49:50.641222 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:50.641153    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x" podUID="8676d045-b8b9-4939-bd4b-089b0d0df9ee" containerName="payload-pre-processing" containerID="cri-o://f1a61f6fcd2b4e982b323a2e6c2acc8eaf0030a0dc9bc3b788b8e66e312d6b2d" gracePeriod=25
Nov 04 03:49:53.628339 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:53.628298    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x" podUID="8676d045-b8b9-4939-bd4b-089b0d0df9ee" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:49:58.361335 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:49:58.361297    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"]
Nov 04 03:49:58.729172 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:49:58.729079    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:50:03.628178 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:03.628128    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x" podUID="8676d045-b8b9-4939-bd4b-089b0d0df9ee" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:50:06.923857 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:06.923818    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"]
Nov 04 03:50:07.035290 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:07.035258    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"]
Nov 04 03:50:10.909923 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:10.909863    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"
Nov 04 03:50:11.037632 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:11.037589    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8676d045-b8b9-4939-bd4b-089b0d0df9ee-kube-api-access-4h24t\" (UniqueName: \"kubernetes.io/projected/8676d045-b8b9-4939-bd4b-089b0d0df9ee-kube-api-access-4h24t\") pod \"8676d045-b8b9-4939-bd4b-089b0d0df9ee\" (UID: \"8676d045-b8b9-4939-bd4b-089b0d0df9ee\") "
Nov 04 03:50:11.037810 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:11.037672    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8676d045-b8b9-4939-bd4b-089b0d0df9ee-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8676d045-b8b9-4939-bd4b-089b0d0df9ee-plugins-config-volume\") pod \"8676d045-b8b9-4939-bd4b-089b0d0df9ee\" (UID: \"8676d045-b8b9-4939-bd4b-089b0d0df9ee\") "
Nov 04 03:50:11.038081 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:11.038044    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8676d045-b8b9-4939-bd4b-089b0d0df9ee-plugins-config-volume" pod "8676d045-b8b9-4939-bd4b-089b0d0df9ee" (UID: "8676d045-b8b9-4939-bd4b-089b0d0df9ee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:50:11.039782 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:11.039757    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8676d045-b8b9-4939-bd4b-089b0d0df9ee-kube-api-access-4h24t" pod "8676d045-b8b9-4939-bd4b-089b0d0df9ee" (UID: "8676d045-b8b9-4939-bd4b-089b0d0df9ee"). InnerVolumeSpecName "kube-api-access-4h24t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:50:11.139473 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:11.139423    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4h24t\" (UniqueName: \"kubernetes.io/projected/8676d045-b8b9-4939-bd4b-089b0d0df9ee-kube-api-access-4h24t\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:11.139473 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:11.139468    4521 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8676d045-b8b9-4939-bd4b-089b0d0df9ee-plugins-config-volume\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:11.595754 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:11.595715    4521 generic.go:356] "Generic (PLEG): container finished" podID="8676d045-b8b9-4939-bd4b-089b0d0df9ee" containerID="f1a61f6fcd2b4e982b323a2e6c2acc8eaf0030a0dc9bc3b788b8e66e312d6b2d" exitCode=0
Nov 04 03:50:11.595954 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:11.595778    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"
Nov 04 03:50:11.595954 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:11.595793    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x" event={"ID":"8676d045-b8b9-4939-bd4b-089b0d0df9ee","Type":"ContainerDied","Data":"f1a61f6fcd2b4e982b323a2e6c2acc8eaf0030a0dc9bc3b788b8e66e312d6b2d"}
Nov 04 03:50:11.595954 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:11.595896    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x" event={"ID":"8676d045-b8b9-4939-bd4b-089b0d0df9ee","Type":"ContainerDied","Data":"17416f9ab9624855ac869538db409902edb3d1c4ac53796066797b9733af7e4a"}
Nov 04 03:50:11.595954 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:11.595927    4521 scope.go:122] "RemoveContainer" containerID="f1a61f6fcd2b4e982b323a2e6c2acc8eaf0030a0dc9bc3b788b8e66e312d6b2d"
Nov 04 03:50:11.605537 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:11.605518    4521 scope.go:122] "RemoveContainer" containerID="f1a61f6fcd2b4e982b323a2e6c2acc8eaf0030a0dc9bc3b788b8e66e312d6b2d"
Nov 04 03:50:11.605804 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:50:11.605786    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1a61f6fcd2b4e982b323a2e6c2acc8eaf0030a0dc9bc3b788b8e66e312d6b2d\": container with ID starting with f1a61f6fcd2b4e982b323a2e6c2acc8eaf0030a0dc9bc3b788b8e66e312d6b2d not found: ID does not exist" containerID="f1a61f6fcd2b4e982b323a2e6c2acc8eaf0030a0dc9bc3b788b8e66e312d6b2d"
Nov 04 03:50:11.605904 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:11.605811    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1a61f6fcd2b4e982b323a2e6c2acc8eaf0030a0dc9bc3b788b8e66e312d6b2d"} err="failed to get container status \"f1a61f6fcd2b4e982b323a2e6c2acc8eaf0030a0dc9bc3b788b8e66e312d6b2d\": rpc error: code = NotFound desc = could not find container \"f1a61f6fcd2b4e982b323a2e6c2acc8eaf0030a0dc9bc3b788b8e66e312d6b2d\": container with ID starting with f1a61f6fcd2b4e982b323a2e6c2acc8eaf0030a0dc9bc3b788b8e66e312d6b2d not found: ID does not exist"
Nov 04 03:50:11.763948 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:11.763916    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"]
Nov 04 03:50:11.771260 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:11.771229    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-951893-6d666ffdd7-t8r4x"]
Nov 04 03:50:11.911250 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:11.911173    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8676d045-b8b9-4939-bd4b-089b0d0df9ee" path="/var/lib/kubelet/pods/8676d045-b8b9-4939-bd4b-089b0d0df9ee/volumes"
Nov 04 03:50:11.924945 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:11.924912    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7" podUID="508106f1-2ca8-4a87-aa92-49d54846e38b" containerName="payload-processing" containerID="cri-o://ed779ff59049849edd9c060251a4be29c6dc3d58888fd9b49786a4dc6d6a7fc5" gracePeriod=25
Nov 04 03:50:12.035926 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.035886    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b" podUID="52ad1670-708f-4b8d-8d2d-4c5101f71319" containerName="payload-pre-processing" containerID="cri-o://4def60bed6cb81c9823da286d4538e61651adf10322c968b3b9d2f6ecf452a3b" gracePeriod=25
Nov 04 03:50:12.396474 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.396437    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"]
Nov 04 03:50:12.396918 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.396903    4521 cpu_manager.go:414] "containerMap: removing container" podUID="53e6e076-af21-469c-9713-6b7f608833bc" containerName="maas-api"
Nov 04 03:50:12.396968 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.396918    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53e6e076-af21-469c-9713-6b7f608833bc" containerName="maas-api"
Nov 04 03:50:12.396968 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.396931    4521 cpu_manager.go:414] "containerMap: removing container" podUID="d4814588-6d91-4621-b701-065525865091" containerName="istio-proxy"
Nov 04 03:50:12.396968 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.396936    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4814588-6d91-4621-b701-065525865091" containerName="istio-proxy"
Nov 04 03:50:12.396968 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.396945    4521 cpu_manager.go:414] "containerMap: removing container" podUID="8676d045-b8b9-4939-bd4b-089b0d0df9ee" containerName="payload-pre-processing"
Nov 04 03:50:12.396968 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.396950    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8676d045-b8b9-4939-bd4b-089b0d0df9ee" containerName="payload-pre-processing"
Nov 04 03:50:12.396968 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.396966    4521 cpu_manager.go:414] "containerMap: removing container" podUID="a9ea7e36-1384-4ab8-9f5c-a5562faba8c2" containerName="maas-api"
Nov 04 03:50:12.396968 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.396970    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9ea7e36-1384-4ab8-9f5c-a5562faba8c2" containerName="maas-api"
Nov 04 03:50:12.397173 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.397049    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="53e6e076-af21-469c-9713-6b7f608833bc" containerName="maas-api"
Nov 04 03:50:12.397173 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.397057    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="8676d045-b8b9-4939-bd4b-089b0d0df9ee" containerName="payload-pre-processing"
Nov 04 03:50:12.397173 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.397064    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4814588-6d91-4621-b701-065525865091" containerName="istio-proxy"
Nov 04 03:50:12.397173 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.397073    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9ea7e36-1384-4ab8-9f5c-a5562faba8c2" containerName="maas-api"
Nov 04 03:50:12.401475 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.401457    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.505369 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.505334    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-fba151-openshift-default-dockercfg-v44lg\""
Nov 04 03:50:12.507676 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.507648    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"]
Nov 04 03:50:12.554209 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.554170    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-credential-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.554387 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.554220    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c69777ad-b490-46c3-a447-ebb504be363b-istio-token\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.554387 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.554249    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-istio-data\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.554387 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.554334    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c69777ad-b490-46c3-a447-ebb504be363b-istio-podinfo\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.554504 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.554411    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-istio-envoy\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.554504 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.554432    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvzqh\" (UniqueName: \"kubernetes.io/projected/c69777ad-b490-46c3-a447-ebb504be363b-kube-api-access-dvzqh\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.554504 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.554460    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-workload-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.554635 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.554545    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c69777ad-b490-46c3-a447-ebb504be363b-istiod-ca-cert\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.554635 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.554564    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-workload-certs\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.655503 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.655412    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c69777ad-b490-46c3-a447-ebb504be363b-istiod-ca-cert\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.655503 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.655451    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-workload-certs\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.655747 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.655510    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-credential-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.655747 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.655540    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c69777ad-b490-46c3-a447-ebb504be363b-istio-token\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.655747 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.655569    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-istio-data\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.655747 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.655596    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c69777ad-b490-46c3-a447-ebb504be363b-istio-podinfo\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.655747 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.655658    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-istio-envoy\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.655747 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.655686    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvzqh\" (UniqueName: \"kubernetes.io/projected/c69777ad-b490-46c3-a447-ebb504be363b-kube-api-access-dvzqh\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.655747 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.655725    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-workload-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.656116 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.656044    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-credential-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.656177 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.656111    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-istio-data\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.656269 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.656248    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-workload-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.656319 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.656267    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-workload-certs\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.656355 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.656320    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c69777ad-b490-46c3-a447-ebb504be363b-istiod-ca-cert\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.658011 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.657991    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-istio-envoy\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.658157 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.658139    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c69777ad-b490-46c3-a447-ebb504be363b-istio-podinfo\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.717099 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.717066    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c69777ad-b490-46c3-a447-ebb504be363b-istio-token\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:12.717248 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:12.717156    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvzqh\" (UniqueName: \"kubernetes.io/projected/c69777ad-b490-46c3-a447-ebb504be363b-kube-api-access-dvzqh\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:13.015019 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.014936    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:13.381715 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.381667    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7" podUID="5932542b-7aff-428d-a8e9-b8a20235035d" containerName="main" containerID="cri-o://02a22995f7a4748e909b94da8e681573122786d0c00fa859235609ceebe8039e" gracePeriod=45
Nov 04 03:50:13.444520 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.444493    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"]
Nov 04 03:50:13.461467 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:50:13.461437    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc69777ad_b490_46c3_a447_ebb504be363b.slice/crio-4ec6966277eef28248a6bb64fa9fb78f79409a3830d7b13b724110c61af71106 WatchSource:0}: Error finding container 4ec6966277eef28248a6bb64fa9fb78f79409a3830d7b13b724110c61af71106: Status 404 returned error can't find the container with id 4ec6966277eef28248a6bb64fa9fb78f79409a3830d7b13b724110c61af71106
Nov 04 03:50:13.486996 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.486955    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:50:13.487127 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.487028    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:50:13.487127 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.487058    4521 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:50:13.611519 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.611489    4521 generic.go:356] "Generic (PLEG): container finished" podID="5932542b-7aff-428d-a8e9-b8a20235035d" containerID="02a22995f7a4748e909b94da8e681573122786d0c00fa859235609ceebe8039e" exitCode=0
Nov 04 03:50:13.611718 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.611578    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7" event={"ID":"5932542b-7aff-428d-a8e9-b8a20235035d","Type":"ContainerDied","Data":"02a22995f7a4748e909b94da8e681573122786d0c00fa859235609ceebe8039e"}
Nov 04 03:50:13.613521 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.613489    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4" event={"ID":"c69777ad-b490-46c3-a447-ebb504be363b","Type":"ContainerStarted","Data":"81da5f428b8b2ae62cb72a38a5944c63929142e6098f537576fbf629da46637c"}
Nov 04 03:50:13.613680 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.613549    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4" event={"ID":"c69777ad-b490-46c3-a447-ebb504be363b","Type":"ContainerStarted","Data":"4ec6966277eef28248a6bb64fa9fb78f79409a3830d7b13b724110c61af71106"}
Nov 04 03:50:13.636559 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.636502    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:50:13.737280 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.737230    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4" podStartSLOduration=1.737216547 podStartE2EDuration="1.737216547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:50:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:50:13.735396464 +0000 UTC m=+1979.939996787" watchObservedRunningTime="2026-11-04 03:50:13.737216547 +0000 UTC m=+1979.941816848"
Nov 04 03:50:13.767697 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.767668    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-tmp-dir\") pod \"5932542b-7aff-428d-a8e9-b8a20235035d\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") "
Nov 04 03:50:13.767884 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.767714    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5932542b-7aff-428d-a8e9-b8a20235035d-kube-api-access-ndbkf\" (UniqueName: \"kubernetes.io/projected/5932542b-7aff-428d-a8e9-b8a20235035d-kube-api-access-ndbkf\") pod \"5932542b-7aff-428d-a8e9-b8a20235035d\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") "
Nov 04 03:50:13.767884 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.767756    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-dshm\") pod \"5932542b-7aff-428d-a8e9-b8a20235035d\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") "
Nov 04 03:50:13.767884 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.767783    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5932542b-7aff-428d-a8e9-b8a20235035d-tls-certs\" (UniqueName: \"kubernetes.io/secret/5932542b-7aff-428d-a8e9-b8a20235035d-tls-certs\") pod \"5932542b-7aff-428d-a8e9-b8a20235035d\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") "
Nov 04 03:50:13.768029 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.767953    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-model-cache\") pod \"5932542b-7aff-428d-a8e9-b8a20235035d\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") "
Nov 04 03:50:13.768029 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.767953    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-tmp-dir" pod "5932542b-7aff-428d-a8e9-b8a20235035d" (UID: "5932542b-7aff-428d-a8e9-b8a20235035d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:13.768029 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.767994    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-home\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-home\") pod \"5932542b-7aff-428d-a8e9-b8a20235035d\" (UID: \"5932542b-7aff-428d-a8e9-b8a20235035d\") "
Nov 04 03:50:13.768314 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.768256    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-model-cache" pod "5932542b-7aff-428d-a8e9-b8a20235035d" (UID: "5932542b-7aff-428d-a8e9-b8a20235035d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:13.768447 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.768313    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-home" pod "5932542b-7aff-428d-a8e9-b8a20235035d" (UID: "5932542b-7aff-428d-a8e9-b8a20235035d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:13.768583 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.768564    4521 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-tmp-dir\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.768656 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.768587    4521 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-model-cache\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.768656 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.768602    4521 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-home\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.770284 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.770258    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5932542b-7aff-428d-a8e9-b8a20235035d-kube-api-access-ndbkf" pod "5932542b-7aff-428d-a8e9-b8a20235035d" (UID: "5932542b-7aff-428d-a8e9-b8a20235035d"). InnerVolumeSpecName "kube-api-access-ndbkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:50:13.770389 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.770307    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5932542b-7aff-428d-a8e9-b8a20235035d-tls-certs" pod "5932542b-7aff-428d-a8e9-b8a20235035d" (UID: "5932542b-7aff-428d-a8e9-b8a20235035d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:50:13.770389 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.770381    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-dshm" pod "5932542b-7aff-428d-a8e9-b8a20235035d" (UID: "5932542b-7aff-428d-a8e9-b8a20235035d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:13.869711 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.869670    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndbkf\" (UniqueName: \"kubernetes.io/projected/5932542b-7aff-428d-a8e9-b8a20235035d-kube-api-access-ndbkf\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.869711 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.869701    4521 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5932542b-7aff-428d-a8e9-b8a20235035d-dshm\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.869711 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:13.869715    4521 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5932542b-7aff-428d-a8e9-b8a20235035d-tls-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:14.017092 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:14.017054    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:14.018644 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:14.018612    4521 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 04 03:50:14.018753 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:14.018676    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4" podUID="c69777ad-b490-46c3-a447-ebb504be363b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 04 03:50:14.315326 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:14.315289    4521 scope.go:122] "RemoveContainer" containerID="02a22995f7a4748e909b94da8e681573122786d0c00fa859235609ceebe8039e"
Nov 04 03:50:14.618209 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:14.618103    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7" event={"ID":"5932542b-7aff-428d-a8e9-b8a20235035d","Type":"ContainerDied","Data":"5d90ee505f747eeebb38b9296e58369f8b1b17e8ae4a99ce70a4748fd038574d"}
Nov 04 03:50:14.618209 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:14.618148    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"
Nov 04 03:50:14.860683 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:14.860641    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"]
Nov 04 03:50:14.865072 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:14.865043    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/distinct-w2-fba151-kserve-6b976ff9f5-pdcf7"]
Nov 04 03:50:15.015523 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:15.015434    4521 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 04 03:50:15.015523 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:15.015501    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4" podUID="c69777ad-b490-46c3-a447-ebb504be363b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 04 03:50:15.916423 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:15.916387    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5932542b-7aff-428d-a8e9-b8a20235035d" path="/var/lib/kubelet/pods/5932542b-7aff-428d-a8e9-b8a20235035d/volumes"
Nov 04 03:50:16.016398 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:16.016362    4521 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 04 03:50:16.016582 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:16.016418    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4" podUID="c69777ad-b490-46c3-a447-ebb504be363b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 04 03:50:17.015648 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:17.015609    4521 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 04 03:50:17.016084 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:17.015684    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4" podUID="c69777ad-b490-46c3-a447-ebb504be363b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 04 03:50:18.015870 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.015805    4521 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 04 03:50:18.016289 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.015901    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4" podUID="c69777ad-b490-46c3-a447-ebb504be363b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 04 03:50:18.198866 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.185587    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"]
Nov 04 03:50:18.199359 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.199323    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47" podUID="3c6d1176-f442-4f20-84e5-f89e0c12f263" containerName="maas-api" containerID="cri-o://3ff5440858dc62872a4a2f511ab019a15343013aac9d20b9978662654fcc1b66" gracePeriod=30
Nov 04 03:50:18.347062 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.347028    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7" podUID="508106f1-2ca8-4a87-aa92-49d54846e38b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:50:18.347344 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.347319    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b" podUID="52ad1670-708f-4b8d-8d2d-4c5101f71319" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:50:18.529174 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.529151    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:50:18.639010 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.638927    4521 generic.go:356] "Generic (PLEG): container finished" podID="3c6d1176-f442-4f20-84e5-f89e0c12f263" containerID="3ff5440858dc62872a4a2f511ab019a15343013aac9d20b9978662654fcc1b66" exitCode=0
Nov 04 03:50:18.639010 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.638976    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"
Nov 04 03:50:18.639010 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.638986    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47" event={"ID":"3c6d1176-f442-4f20-84e5-f89e0c12f263","Type":"ContainerDied","Data":"3ff5440858dc62872a4a2f511ab019a15343013aac9d20b9978662654fcc1b66"}
Nov 04 03:50:18.639264 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.639030    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47" event={"ID":"3c6d1176-f442-4f20-84e5-f89e0c12f263","Type":"ContainerDied","Data":"6c74f5ae841059374814918dbbea1fe6e5ed4301e746c5d6b9ac6161b0db90c5"}
Nov 04 03:50:18.639264 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.639052    4521 scope.go:122] "RemoveContainer" containerID="3ff5440858dc62872a4a2f511ab019a15343013aac9d20b9978662654fcc1b66"
Nov 04 03:50:18.641273 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.641251    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c6d1176-f442-4f20-84e5-f89e0c12f263-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c6d1176-f442-4f20-84e5-f89e0c12f263-openshift-service-ca\") pod \"3c6d1176-f442-4f20-84e5-f89e0c12f263\" (UID: \"3c6d1176-f442-4f20-84e5-f89e0c12f263\") "
Nov 04 03:50:18.641404 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.641359    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3c6d1176-f442-4f20-84e5-f89e0c12f263-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c6d1176-f442-4f20-84e5-f89e0c12f263-metrics-certs\") pod \"3c6d1176-f442-4f20-84e5-f89e0c12f263\" (UID: \"3c6d1176-f442-4f20-84e5-f89e0c12f263\") "
Nov 04 03:50:18.641470 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.641400    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3c6d1176-f442-4f20-84e5-f89e0c12f263-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c6d1176-f442-4f20-84e5-f89e0c12f263-maas-api-tls\") pod \"3c6d1176-f442-4f20-84e5-f89e0c12f263\" (UID: \"3c6d1176-f442-4f20-84e5-f89e0c12f263\") "
Nov 04 03:50:18.641543 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.641528    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c6d1176-f442-4f20-84e5-f89e0c12f263-kube-api-access-m7qzp\" (UniqueName: \"kubernetes.io/projected/3c6d1176-f442-4f20-84e5-f89e0c12f263-kube-api-access-m7qzp\") pod \"3c6d1176-f442-4f20-84e5-f89e0c12f263\" (UID: \"3c6d1176-f442-4f20-84e5-f89e0c12f263\") "
Nov 04 03:50:18.641647 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.641622    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c6d1176-f442-4f20-84e5-f89e0c12f263-openshift-service-ca" pod "3c6d1176-f442-4f20-84e5-f89e0c12f263" (UID: "3c6d1176-f442-4f20-84e5-f89e0c12f263"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:50:18.641962 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.641938    4521 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c6d1176-f442-4f20-84e5-f89e0c12f263-openshift-service-ca\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:18.643780 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.643754    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c6d1176-f442-4f20-84e5-f89e0c12f263-metrics-certs" pod "3c6d1176-f442-4f20-84e5-f89e0c12f263" (UID: "3c6d1176-f442-4f20-84e5-f89e0c12f263"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:50:18.643898 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.643826    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c6d1176-f442-4f20-84e5-f89e0c12f263-maas-api-tls" pod "3c6d1176-f442-4f20-84e5-f89e0c12f263" (UID: "3c6d1176-f442-4f20-84e5-f89e0c12f263"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:50:18.645353 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.645328    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c6d1176-f442-4f20-84e5-f89e0c12f263-kube-api-access-m7qzp" pod "3c6d1176-f442-4f20-84e5-f89e0c12f263" (UID: "3c6d1176-f442-4f20-84e5-f89e0c12f263"). InnerVolumeSpecName "kube-api-access-m7qzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:50:18.669411 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.669387    4521 scope.go:122] "RemoveContainer" containerID="3ff5440858dc62872a4a2f511ab019a15343013aac9d20b9978662654fcc1b66"
Nov 04 03:50:18.669787 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:50:18.669765    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ff5440858dc62872a4a2f511ab019a15343013aac9d20b9978662654fcc1b66\": container with ID starting with 3ff5440858dc62872a4a2f511ab019a15343013aac9d20b9978662654fcc1b66 not found: ID does not exist" containerID="3ff5440858dc62872a4a2f511ab019a15343013aac9d20b9978662654fcc1b66"
Nov 04 03:50:18.669887 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.669795    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ff5440858dc62872a4a2f511ab019a15343013aac9d20b9978662654fcc1b66"} err="failed to get container status \"3ff5440858dc62872a4a2f511ab019a15343013aac9d20b9978662654fcc1b66\": rpc error: code = NotFound desc = could not find container \"3ff5440858dc62872a4a2f511ab019a15343013aac9d20b9978662654fcc1b66\": container with ID starting with 3ff5440858dc62872a4a2f511ab019a15343013aac9d20b9978662654fcc1b66 not found: ID does not exist"
Nov 04 03:50:18.743038 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.742994    4521 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c6d1176-f442-4f20-84e5-f89e0c12f263-metrics-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:18.743038 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.743028    4521 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c6d1176-f442-4f20-84e5-f89e0c12f263-maas-api-tls\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:18.743038 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:18.743041    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7qzp\" (UniqueName: \"kubernetes.io/projected/3c6d1176-f442-4f20-84e5-f89e0c12f263-kube-api-access-m7qzp\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:19.015506 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:19.015466    4521 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 04 03:50:19.015702 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:19.015523    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4" podUID="c69777ad-b490-46c3-a447-ebb504be363b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 04 03:50:19.069836 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:19.069801    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"]
Nov 04 03:50:19.076481 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:19.076441    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-fba151-746847b655-dxf47"]
Nov 04 03:50:19.911639 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:19.911599    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c6d1176-f442-4f20-84e5-f89e0c12f263" path="/var/lib/kubelet/pods/3c6d1176-f442-4f20-84e5-f89e0c12f263/volumes"
Nov 04 03:50:20.016267 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:20.016226    4521 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 04 03:50:20.016451 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:20.016286    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4" podUID="c69777ad-b490-46c3-a447-ebb504be363b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 04 03:50:22.057029 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:22.056985    4521 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:50:22.057510 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:22.057055    4521 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4" podUID="c69777ad-b490-46c3-a447-ebb504be363b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:50:22.159517 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:22.159485    4521 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:22.159903 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:22.159878    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:22.161384 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:22.161361    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:28.346806 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:28.346766    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b" podUID="52ad1670-708f-4b8d-8d2d-4c5101f71319" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:50:28.346806 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:28.346791    4521 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7" podUID="508106f1-2ca8-4a87-aa92-49d54846e38b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:50:28.729894 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:50:28.729795    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:50:32.201530 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.201498    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"
Nov 04 03:50:32.285988 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.285958    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/508106f1-2ca8-4a87-aa92-49d54846e38b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/508106f1-2ca8-4a87-aa92-49d54846e38b-plugins-config-volume\") pod \"508106f1-2ca8-4a87-aa92-49d54846e38b\" (UID: \"508106f1-2ca8-4a87-aa92-49d54846e38b\") "
Nov 04 03:50:32.286130 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.286103    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/508106f1-2ca8-4a87-aa92-49d54846e38b-kube-api-access-ctwhx\" (UniqueName: \"kubernetes.io/projected/508106f1-2ca8-4a87-aa92-49d54846e38b-kube-api-access-ctwhx\") pod \"508106f1-2ca8-4a87-aa92-49d54846e38b\" (UID: \"508106f1-2ca8-4a87-aa92-49d54846e38b\") "
Nov 04 03:50:32.286338 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.286302    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/508106f1-2ca8-4a87-aa92-49d54846e38b-plugins-config-volume" pod "508106f1-2ca8-4a87-aa92-49d54846e38b" (UID: "508106f1-2ca8-4a87-aa92-49d54846e38b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:50:32.286517 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.286493    4521 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/508106f1-2ca8-4a87-aa92-49d54846e38b-plugins-config-volume\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:32.288180 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.288151    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/508106f1-2ca8-4a87-aa92-49d54846e38b-kube-api-access-ctwhx" pod "508106f1-2ca8-4a87-aa92-49d54846e38b" (UID: "508106f1-2ca8-4a87-aa92-49d54846e38b"). InnerVolumeSpecName "kube-api-access-ctwhx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:50:32.302976 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.302958    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"
Nov 04 03:50:32.387498 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.387462    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52ad1670-708f-4b8d-8d2d-4c5101f71319-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52ad1670-708f-4b8d-8d2d-4c5101f71319-plugins-config-volume\") pod \"52ad1670-708f-4b8d-8d2d-4c5101f71319\" (UID: \"52ad1670-708f-4b8d-8d2d-4c5101f71319\") "
Nov 04 03:50:32.387498 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.387502    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52ad1670-708f-4b8d-8d2d-4c5101f71319-kube-api-access-2rpln\" (UniqueName: \"kubernetes.io/projected/52ad1670-708f-4b8d-8d2d-4c5101f71319-kube-api-access-2rpln\") pod \"52ad1670-708f-4b8d-8d2d-4c5101f71319\" (UID: \"52ad1670-708f-4b8d-8d2d-4c5101f71319\") "
Nov 04 03:50:32.387790 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.387777    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctwhx\" (UniqueName: \"kubernetes.io/projected/508106f1-2ca8-4a87-aa92-49d54846e38b-kube-api-access-ctwhx\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:32.387955 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.387919    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52ad1670-708f-4b8d-8d2d-4c5101f71319-plugins-config-volume" pod "52ad1670-708f-4b8d-8d2d-4c5101f71319" (UID: "52ad1670-708f-4b8d-8d2d-4c5101f71319"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:50:32.389638 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.389620    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52ad1670-708f-4b8d-8d2d-4c5101f71319-kube-api-access-2rpln" pod "52ad1670-708f-4b8d-8d2d-4c5101f71319" (UID: "52ad1670-708f-4b8d-8d2d-4c5101f71319"). InnerVolumeSpecName "kube-api-access-2rpln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:50:32.488505 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.488433    4521 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52ad1670-708f-4b8d-8d2d-4c5101f71319-plugins-config-volume\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:32.488505 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.488460    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rpln\" (UniqueName: \"kubernetes.io/projected/52ad1670-708f-4b8d-8d2d-4c5101f71319-kube-api-access-2rpln\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:32.710198 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.710158    4521 generic.go:356] "Generic (PLEG): container finished" podID="508106f1-2ca8-4a87-aa92-49d54846e38b" containerID="ed779ff59049849edd9c060251a4be29c6dc3d58888fd9b49786a4dc6d6a7fc5" exitCode=0
Nov 04 03:50:32.710350 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.710207    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"
Nov 04 03:50:32.710350 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.710244    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7" event={"ID":"508106f1-2ca8-4a87-aa92-49d54846e38b","Type":"ContainerDied","Data":"ed779ff59049849edd9c060251a4be29c6dc3d58888fd9b49786a4dc6d6a7fc5"}
Nov 04 03:50:32.710350 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.710340    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7" event={"ID":"508106f1-2ca8-4a87-aa92-49d54846e38b","Type":"ContainerDied","Data":"26ad7074966720ae8051affd15b2399460820efa4d9d66db6485653d1ce7b2d9"}
Nov 04 03:50:32.710493 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.710375    4521 scope.go:122] "RemoveContainer" containerID="ed779ff59049849edd9c060251a4be29c6dc3d58888fd9b49786a4dc6d6a7fc5"
Nov 04 03:50:32.711692 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.711672    4521 generic.go:356] "Generic (PLEG): container finished" podID="52ad1670-708f-4b8d-8d2d-4c5101f71319" containerID="4def60bed6cb81c9823da286d4538e61651adf10322c968b3b9d2f6ecf452a3b" exitCode=0
Nov 04 03:50:32.711802 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.711731    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"
Nov 04 03:50:32.711872 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.711733    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b" event={"ID":"52ad1670-708f-4b8d-8d2d-4c5101f71319","Type":"ContainerDied","Data":"4def60bed6cb81c9823da286d4538e61651adf10322c968b3b9d2f6ecf452a3b"}
Nov 04 03:50:32.711872 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.711859    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b" event={"ID":"52ad1670-708f-4b8d-8d2d-4c5101f71319","Type":"ContainerDied","Data":"ed4dd8251fdf2a1290b91924297c42631b17f7fffa0eb28e68562a8e0b7d5350"}
Nov 04 03:50:32.720306 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.720285    4521 scope.go:122] "RemoveContainer" containerID="ed779ff59049849edd9c060251a4be29c6dc3d58888fd9b49786a4dc6d6a7fc5"
Nov 04 03:50:32.720573 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:50:32.720552    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed779ff59049849edd9c060251a4be29c6dc3d58888fd9b49786a4dc6d6a7fc5\": container with ID starting with ed779ff59049849edd9c060251a4be29c6dc3d58888fd9b49786a4dc6d6a7fc5 not found: ID does not exist" containerID="ed779ff59049849edd9c060251a4be29c6dc3d58888fd9b49786a4dc6d6a7fc5"
Nov 04 03:50:32.720668 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.720584    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed779ff59049849edd9c060251a4be29c6dc3d58888fd9b49786a4dc6d6a7fc5"} err="failed to get container status \"ed779ff59049849edd9c060251a4be29c6dc3d58888fd9b49786a4dc6d6a7fc5\": rpc error: code = NotFound desc = could not find container \"ed779ff59049849edd9c060251a4be29c6dc3d58888fd9b49786a4dc6d6a7fc5\": container with ID starting with ed779ff59049849edd9c060251a4be29c6dc3d58888fd9b49786a4dc6d6a7fc5 not found: ID does not exist"
Nov 04 03:50:32.720668 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.720606    4521 scope.go:122] "RemoveContainer" containerID="4def60bed6cb81c9823da286d4538e61651adf10322c968b3b9d2f6ecf452a3b"
Nov 04 03:50:32.728585 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.728568    4521 scope.go:122] "RemoveContainer" containerID="4def60bed6cb81c9823da286d4538e61651adf10322c968b3b9d2f6ecf452a3b"
Nov 04 03:50:32.728861 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:50:32.728826    4521 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4def60bed6cb81c9823da286d4538e61651adf10322c968b3b9d2f6ecf452a3b\": container with ID starting with 4def60bed6cb81c9823da286d4538e61651adf10322c968b3b9d2f6ecf452a3b not found: ID does not exist" containerID="4def60bed6cb81c9823da286d4538e61651adf10322c968b3b9d2f6ecf452a3b"
Nov 04 03:50:32.728920 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.728867    4521 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4def60bed6cb81c9823da286d4538e61651adf10322c968b3b9d2f6ecf452a3b"} err="failed to get container status \"4def60bed6cb81c9823da286d4538e61651adf10322c968b3b9d2f6ecf452a3b\": rpc error: code = NotFound desc = could not find container \"4def60bed6cb81c9823da286d4538e61651adf10322c968b3b9d2f6ecf452a3b\": container with ID starting with 4def60bed6cb81c9823da286d4538e61651adf10322c968b3b9d2f6ecf452a3b not found: ID does not exist"
Nov 04 03:50:32.876895 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.876863    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"]
Nov 04 03:50:32.926940 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.926906    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-fba151-6ccbb6bc67-j2n4b"]
Nov 04 03:50:32.993489 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.993457    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"]
Nov 04 03:50:32.997542 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:32.997515    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-fba151-59d7cc774-nw6l7"]
Nov 04 03:50:33.911811 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:33.911773    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="508106f1-2ca8-4a87-aa92-49d54846e38b" path="/var/lib/kubelet/pods/508106f1-2ca8-4a87-aa92-49d54846e38b/volumes"
Nov 04 03:50:33.912322 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:33.912304    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="52ad1670-708f-4b8d-8d2d-4c5101f71319" path="/var/lib/kubelet/pods/52ad1670-708f-4b8d-8d2d-4c5101f71319/volumes"
Nov 04 03:50:40.546968 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:40.546930    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"]
Nov 04 03:50:40.547444 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:40.547201    4521 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4" podUID="c69777ad-b490-46c3-a447-ebb504be363b" containerName="istio-proxy" containerID="cri-o://81da5f428b8b2ae62cb72a38a5944c63929142e6098f537576fbf629da46637c" gracePeriod=30
Nov 04 03:50:45.784068 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.783885    4521 generic.go:356] "Generic (PLEG): container finished" podID="c69777ad-b490-46c3-a447-ebb504be363b" containerID="81da5f428b8b2ae62cb72a38a5944c63929142e6098f537576fbf629da46637c" exitCode=0
Nov 04 03:50:45.784068 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.783939    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4" event={"ID":"c69777ad-b490-46c3-a447-ebb504be363b","Type":"ContainerDied","Data":"81da5f428b8b2ae62cb72a38a5944c63929142e6098f537576fbf629da46637c"}
Nov 04 03:50:45.807523 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.807497    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:45.927453 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.927375    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-istio-envoy\") pod \"c69777ad-b490-46c3-a447-ebb504be363b\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") "
Nov 04 03:50:45.927453 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.927417    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c69777ad-b490-46c3-a447-ebb504be363b-istio-token\" (UniqueName: \"kubernetes.io/projected/c69777ad-b490-46c3-a447-ebb504be363b-istio-token\") pod \"c69777ad-b490-46c3-a447-ebb504be363b\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") "
Nov 04 03:50:45.927680 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.927455    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c69777ad-b490-46c3-a447-ebb504be363b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c69777ad-b490-46c3-a447-ebb504be363b-istiod-ca-cert\") pod \"c69777ad-b490-46c3-a447-ebb504be363b\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") "
Nov 04 03:50:45.927680 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.927498    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-workload-certs\") pod \"c69777ad-b490-46c3-a447-ebb504be363b\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") "
Nov 04 03:50:45.927680 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.927521    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c69777ad-b490-46c3-a447-ebb504be363b-kube-api-access-dvzqh\" (UniqueName: \"kubernetes.io/projected/c69777ad-b490-46c3-a447-ebb504be363b-kube-api-access-dvzqh\") pod \"c69777ad-b490-46c3-a447-ebb504be363b\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") "
Nov 04 03:50:45.927680 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.927577    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c69777ad-b490-46c3-a447-ebb504be363b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c69777ad-b490-46c3-a447-ebb504be363b-istio-podinfo\") pod \"c69777ad-b490-46c3-a447-ebb504be363b\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") "
Nov 04 03:50:45.927680 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.927612    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-istio-data\") pod \"c69777ad-b490-46c3-a447-ebb504be363b\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") "
Nov 04 03:50:45.927680 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.927648    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-credential-socket\") pod \"c69777ad-b490-46c3-a447-ebb504be363b\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") "
Nov 04 03:50:45.928038 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.927708    4521 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-workload-socket\") pod \"c69777ad-b490-46c3-a447-ebb504be363b\" (UID: \"c69777ad-b490-46c3-a447-ebb504be363b\") "
Nov 04 03:50:45.928038 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.927864    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-workload-certs" pod "c69777ad-b490-46c3-a447-ebb504be363b" (UID: "c69777ad-b490-46c3-a447-ebb504be363b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:45.928038 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.927975    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c69777ad-b490-46c3-a447-ebb504be363b-istiod-ca-cert" pod "c69777ad-b490-46c3-a447-ebb504be363b" (UID: "c69777ad-b490-46c3-a447-ebb504be363b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:50:45.928199 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.928152    4521 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c69777ad-b490-46c3-a447-ebb504be363b-istiod-ca-cert\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:45.928199 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.928174    4521 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-workload-certs\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:45.928459 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.928440    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-istio-data" pod "c69777ad-b490-46c3-a447-ebb504be363b" (UID: "c69777ad-b490-46c3-a447-ebb504be363b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:45.928539 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.928489    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-workload-socket" pod "c69777ad-b490-46c3-a447-ebb504be363b" (UID: "c69777ad-b490-46c3-a447-ebb504be363b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:45.929363 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.929323    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-credential-socket" pod "c69777ad-b490-46c3-a447-ebb504be363b" (UID: "c69777ad-b490-46c3-a447-ebb504be363b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:45.930062 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.929990    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c69777ad-b490-46c3-a447-ebb504be363b-istio-token" pod "c69777ad-b490-46c3-a447-ebb504be363b" (UID: "c69777ad-b490-46c3-a447-ebb504be363b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:50:45.930176 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.930096    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c69777ad-b490-46c3-a447-ebb504be363b-kube-api-access-dvzqh" pod "c69777ad-b490-46c3-a447-ebb504be363b" (UID: "c69777ad-b490-46c3-a447-ebb504be363b"). InnerVolumeSpecName "kube-api-access-dvzqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:50:45.930176 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.930099    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-istio-envoy" pod "c69777ad-b490-46c3-a447-ebb504be363b" (UID: "c69777ad-b490-46c3-a447-ebb504be363b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:45.930422 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:45.930399    4521 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c69777ad-b490-46c3-a447-ebb504be363b-istio-podinfo" pod "c69777ad-b490-46c3-a447-ebb504be363b" (UID: "c69777ad-b490-46c3-a447-ebb504be363b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:50:46.029084 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:46.029048    4521 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-credential-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:46.029084 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:46.029085    4521 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-workload-socket\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:46.029084 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:46.029100    4521 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-istio-envoy\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:46.029343 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:46.029115    4521 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c69777ad-b490-46c3-a447-ebb504be363b-istio-token\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:46.029343 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:46.029130    4521 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvzqh\" (UniqueName: \"kubernetes.io/projected/c69777ad-b490-46c3-a447-ebb504be363b-kube-api-access-dvzqh\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:46.029343 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:46.029144    4521 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c69777ad-b490-46c3-a447-ebb504be363b-istio-podinfo\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:46.029343 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:46.029158    4521 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c69777ad-b490-46c3-a447-ebb504be363b-istio-data\") on node \"ip-10-0-135-242.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:46.790642 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:46.790610    4521 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"
Nov 04 03:50:46.791165 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:46.790646    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4" event={"ID":"c69777ad-b490-46c3-a447-ebb504be363b","Type":"ContainerDied","Data":"4ec6966277eef28248a6bb64fa9fb78f79409a3830d7b13b724110c61af71106"}
Nov 04 03:50:46.791165 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:46.790711    4521 scope.go:122] "RemoveContainer" containerID="81da5f428b8b2ae62cb72a38a5944c63929142e6098f537576fbf629da46637c"
Nov 04 03:50:47.008002 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:47.007958    4521 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"]
Nov 04 03:50:47.058301 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:47.058222    4521 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-kzcq4"]
Nov 04 03:50:47.916774 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:47.916736    4521 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c69777ad-b490-46c3-a447-ebb504be363b" path="/var/lib/kubelet/pods/c69777ad-b490-46c3-a447-ebb504be363b/volumes"
Nov 04 03:50:52.752050 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:50:52.752017    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-hb9vr_bd98323d-8102-4f46-8765-c7ee20226eb7/kuadrant-console-plugin/0.log"
Nov 04 03:50:58.729514 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:50:58.729426    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:51:07.463284 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:51:07.463252    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7cd9b8cccb-7bml5_8af2ef47-8b2a-4065-ad7f-09e70bed85e7/manager/0.log"
Nov 04 03:51:09.656438 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:51:09.656386    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-n2wm8_9754c3f0-c1b7-431d-844e-0536a6654972/manager/0.log"
Nov 04 03:51:28.729870 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:51:28.729806    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:51:46.263116 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:51:46.263081    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7cd9b8cccb-7bml5_8af2ef47-8b2a-4065-ad7f-09e70bed85e7/manager/0.log"
Nov 04 03:51:48.466634 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:51:48.466601    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-n2wm8_9754c3f0-c1b7-431d-844e-0536a6654972/manager/0.log"
Nov 04 03:51:50.068099 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:51:50.068040    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-cbd8487c-8dvdn_ccda3357-d0c9-467f-b765-7cb9d9d5a532/maas-api/0.log"
Nov 04 03:51:54.044481 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:51:54.044453    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-hb9vr_bd98323d-8102-4f46-8765-c7ee20226eb7/kuadrant-console-plugin/0.log"
Nov 04 03:51:57.076794 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:51:57.076762    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7659c7d778-65h9l_da5d1d62-7e67-423b-95ac-8c809a64959c/kube-auth-proxy/0.log"
Nov 04 03:51:57.761391 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:51:57.761362    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-dh692_ba2f1b97-4d8d-4b69-bf55-531600544660/istio-proxy/0.log"
Nov 04 03:51:58.305270 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:51:58.305240    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-fms7t_4a66091b-11c6-4dcb-8870-0ededd2187d6/istio-proxy/0.log"
Nov 04 03:51:58.729563 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:51:58.729476    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:51:59.282550 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:51:59.282501    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-sr9mv_a71c9284-671b-4ed4-8aac-7687a2e3eefe/istio-proxy/0.log"
Nov 04 03:51:59.507746 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:51:59.507709    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-st9jq_beb6c720-f8d2-4337-8faa-c9e5e6414d33/istio-proxy/0.log"
Nov 04 03:52:00.305360 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:00.305310    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-chld2_d770e48e-5587-4da6-838b-f7b52d18c86a/payload-pre-processing/0.log"
Nov 04 03:52:00.536445 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:00.536393    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-khhp6_cabd37d3-8208-4220-80c4-2d28ed781ce3/payload-processing/0.log"
Nov 04 03:52:02.110405 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:02.110371    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-dr574_f1d543fc-fedf-4c20-81b7-69f1b4689eb7/main/0.log"
Nov 04 03:52:02.383163 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:02.383089    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-mgqz5_0030652f-1a8f-42ff-a812-96f235cef9e7/main/0.log"
Nov 04 03:52:02.599195 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:02.599161    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-r4s25_dae99ce4-da89-4503-bc8f-dffb4f800846/main/0.log"
Nov 04 03:52:02.810179 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:02.810141    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-55lw7_6d0dd607-346c-478b-b28e-7738d56b325a/main/0.log"
Nov 04 03:52:03.035209 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:03.035182    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mtl49_61e73af7-b506-42f5-8594-8ad1387e39d5/main/0.log"
Nov 04 03:52:03.547935 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:03.547906    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-4t22t_bee1a355-b980-49a4-acbc-3527d6107ddc/main/0.log"
Nov 04 03:52:03.766935 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:03.766905    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-6j8cv_9c4b4d5b-04ce-4860-90e2-3161029b2ca0/main/0.log"
Nov 04 03:52:13.652372 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.652332    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x26lf/must-gather-rrvct"]
Nov 04 03:52:13.654709 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.652782    4521 cpu_manager.go:414] "containerMap: removing container" podUID="3c6d1176-f442-4f20-84e5-f89e0c12f263" containerName="maas-api"
Nov 04 03:52:13.654709 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.652794    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c6d1176-f442-4f20-84e5-f89e0c12f263" containerName="maas-api"
Nov 04 03:52:13.654709 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.652829    4521 cpu_manager.go:414] "containerMap: removing container" podUID="52ad1670-708f-4b8d-8d2d-4c5101f71319" containerName="payload-pre-processing"
Nov 04 03:52:13.654709 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.652835    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52ad1670-708f-4b8d-8d2d-4c5101f71319" containerName="payload-pre-processing"
Nov 04 03:52:13.654709 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.652861    4521 cpu_manager.go:414] "containerMap: removing container" podUID="c69777ad-b490-46c3-a447-ebb504be363b" containerName="istio-proxy"
Nov 04 03:52:13.654709 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.652869    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c69777ad-b490-46c3-a447-ebb504be363b" containerName="istio-proxy"
Nov 04 03:52:13.654709 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.652878    4521 cpu_manager.go:414] "containerMap: removing container" podUID="508106f1-2ca8-4a87-aa92-49d54846e38b" containerName="payload-processing"
Nov 04 03:52:13.654709 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.652882    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="508106f1-2ca8-4a87-aa92-49d54846e38b" containerName="payload-processing"
Nov 04 03:52:13.654709 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.652888    4521 cpu_manager.go:414] "containerMap: removing container" podUID="5932542b-7aff-428d-a8e9-b8a20235035d" containerName="main"
Nov 04 03:52:13.654709 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.652893    4521 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5932542b-7aff-428d-a8e9-b8a20235035d" containerName="main"
Nov 04 03:52:13.654709 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.652961    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="52ad1670-708f-4b8d-8d2d-4c5101f71319" containerName="payload-pre-processing"
Nov 04 03:52:13.654709 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.652968    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="c69777ad-b490-46c3-a447-ebb504be363b" containerName="istio-proxy"
Nov 04 03:52:13.654709 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.652974    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="508106f1-2ca8-4a87-aa92-49d54846e38b" containerName="payload-processing"
Nov 04 03:52:13.654709 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.652980    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="5932542b-7aff-428d-a8e9-b8a20235035d" containerName="main"
Nov 04 03:52:13.654709 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.652989    4521 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c6d1176-f442-4f20-84e5-f89e0c12f263" containerName="maas-api"
Nov 04 03:52:13.656040 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.656023    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x26lf/must-gather-rrvct"
Nov 04 03:52:13.713630 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.713592    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9b2r\" (UniqueName: \"kubernetes.io/projected/d6269dd0-c624-4643-9f02-289b895b9b6f-kube-api-access-j9b2r\") pod \"must-gather-rrvct\" (UID: \"d6269dd0-c624-4643-9f02-289b895b9b6f\") " pod="openshift-must-gather-x26lf/must-gather-rrvct"
Nov 04 03:52:13.714244 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.713814    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d6269dd0-c624-4643-9f02-289b895b9b6f-must-gather-output\") pod \"must-gather-rrvct\" (UID: \"d6269dd0-c624-4643-9f02-289b895b9b6f\") " pod="openshift-must-gather-x26lf/must-gather-rrvct"
Nov 04 03:52:13.716031 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.716008    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x26lf/must-gather-rrvct"]
Nov 04 03:52:13.760377 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.760343    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x26lf\"/\"openshift-service-ca.crt\""
Nov 04 03:52:13.760377 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.760353    4521 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x26lf\"/\"kube-root-ca.crt\""
Nov 04 03:52:13.760582 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.760361    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-x26lf\"/\"default-dockercfg-l6db9\""
Nov 04 03:52:13.814647 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.814609    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9b2r\" (UniqueName: \"kubernetes.io/projected/d6269dd0-c624-4643-9f02-289b895b9b6f-kube-api-access-j9b2r\") pod \"must-gather-rrvct\" (UID: \"d6269dd0-c624-4643-9f02-289b895b9b6f\") " pod="openshift-must-gather-x26lf/must-gather-rrvct"
Nov 04 03:52:13.814867 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.814735    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d6269dd0-c624-4643-9f02-289b895b9b6f-must-gather-output\") pod \"must-gather-rrvct\" (UID: \"d6269dd0-c624-4643-9f02-289b895b9b6f\") " pod="openshift-must-gather-x26lf/must-gather-rrvct"
Nov 04 03:52:13.815116 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.815093    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d6269dd0-c624-4643-9f02-289b895b9b6f-must-gather-output\") pod \"must-gather-rrvct\" (UID: \"d6269dd0-c624-4643-9f02-289b895b9b6f\") " pod="openshift-must-gather-x26lf/must-gather-rrvct"
Nov 04 03:52:13.876228 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:13.876196    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9b2r\" (UniqueName: \"kubernetes.io/projected/d6269dd0-c624-4643-9f02-289b895b9b6f-kube-api-access-j9b2r\") pod \"must-gather-rrvct\" (UID: \"d6269dd0-c624-4643-9f02-289b895b9b6f\") " pod="openshift-must-gather-x26lf/must-gather-rrvct"
Nov 04 03:52:14.021766 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:14.021742    4521 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-x26lf\"/\"default-dockercfg-l6db9\""
Nov 04 03:52:14.029910 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:14.029889    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x26lf/must-gather-rrvct"
Nov 04 03:52:14.402522 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:14.402468    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x26lf/must-gather-rrvct"]
Nov 04 03:52:14.408712 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:52:14.408678    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd6269dd0_c624_4643_9f02_289b895b9b6f.slice/crio-761f3d8bd680d1f9bf442b7e93eb79c5a423f6a54480cf9d406b7f2b96bc9e86 WatchSource:0}: Error finding container 761f3d8bd680d1f9bf442b7e93eb79c5a423f6a54480cf9d406b7f2b96bc9e86: Status 404 returned error can't find the container with id 761f3d8bd680d1f9bf442b7e93eb79c5a423f6a54480cf9d406b7f2b96bc9e86
Nov 04 03:52:14.411080 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:14.411062    4521 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:52:15.207813 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:15.207764    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x26lf/must-gather-rrvct" event={"ID":"d6269dd0-c624-4643-9f02-289b895b9b6f","Type":"ContainerStarted","Data":"761f3d8bd680d1f9bf442b7e93eb79c5a423f6a54480cf9d406b7f2b96bc9e86"}
Nov 04 03:52:17.221621 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:17.221581    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x26lf/must-gather-rrvct" event={"ID":"d6269dd0-c624-4643-9f02-289b895b9b6f","Type":"ContainerStarted","Data":"3480d59f8c47056810e14e36629a7807f4ae03d3e4dc93cda9e87a29ca8d5a12"}
Nov 04 03:52:17.221621 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:17.221629    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x26lf/must-gather-rrvct" event={"ID":"d6269dd0-c624-4643-9f02-289b895b9b6f","Type":"ContainerStarted","Data":"6a6e4b3a20ab4fb57e512b0ccc773d20501298cb9cf4c5faca58e2e1625a44e8"}
Nov 04 03:52:17.338168 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:17.338106    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-x26lf/must-gather-rrvct" podStartSLOduration=2.430321859 podStartE2EDuration="4.33808664s" totalImagesPullingTime="1.907764781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:52:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:52:17.338026709 +0000 UTC m=+2103.542627012" watchObservedRunningTime="2026-11-04 03:52:17.33808664 +0000 UTC m=+2103.542686937"
Nov 04 03:52:23.451207 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:23.451173    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qbcj2_0924a641-d0e7-4cdb-8130-f17a47a4c026/global-pull-secret-syncer/0.log"
Nov 04 03:52:25.158231 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:25.158200    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7cd9b8cccb-7bml5_8af2ef47-8b2a-4065-ad7f-09e70bed85e7/manager/0.log"
Nov 04 03:52:27.381468 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:27.381109    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-n2wm8_9754c3f0-c1b7-431d-844e-0536a6654972/manager/0.log"
Nov 04 03:52:27.939342 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:27.939307    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-sd8pw_7e6f6098-4287-4076-a8c5-2e55067dfb87/konnectivity-agent/0.log"
Nov 04 03:52:28.749435 ip-10-0-135-242 kubenswrapper[4521]: E1104 03:52:28.749393    4521 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-t5zsk" containerName="extractor"
Nov 04 03:52:29.272970 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:29.272937    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-242.us-west-2.compute.internal_172fe89364b7e525fac46ad33508ae4e/haproxy/0.log"
Nov 04 03:52:30.458595 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:30.458559    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7cd9b8cccb-7bml5_8af2ef47-8b2a-4065-ad7f-09e70bed85e7/manager/0.log"
Nov 04 03:52:32.560058 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:32.560025    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-n2wm8_9754c3f0-c1b7-431d-844e-0536a6654972/manager/0.log"
Nov 04 03:52:33.720047 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.720011    4521 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"]
Nov 04 03:52:33.725638 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.725606    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:33.771030 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.770995    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f926f149-ef3d-4d31-a1fb-de1912c2c5d1-proc\") pod \"perf-node-gather-daemonset-b7txh\" (UID: \"f926f149-ef3d-4d31-a1fb-de1912c2c5d1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:33.771197 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.771067    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f926f149-ef3d-4d31-a1fb-de1912c2c5d1-lib-modules\") pod \"perf-node-gather-daemonset-b7txh\" (UID: \"f926f149-ef3d-4d31-a1fb-de1912c2c5d1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:33.771197 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.771093    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpd2f\" (UniqueName: \"kubernetes.io/projected/f926f149-ef3d-4d31-a1fb-de1912c2c5d1-kube-api-access-zpd2f\") pod \"perf-node-gather-daemonset-b7txh\" (UID: \"f926f149-ef3d-4d31-a1fb-de1912c2c5d1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:33.771197 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.771150    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f926f149-ef3d-4d31-a1fb-de1912c2c5d1-podres\") pod \"perf-node-gather-daemonset-b7txh\" (UID: \"f926f149-ef3d-4d31-a1fb-de1912c2c5d1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:33.771303 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.771219    4521 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f926f149-ef3d-4d31-a1fb-de1912c2c5d1-sys\") pod \"perf-node-gather-daemonset-b7txh\" (UID: \"f926f149-ef3d-4d31-a1fb-de1912c2c5d1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:33.817167 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.817134    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"]
Nov 04 03:52:33.874700 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.874662    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f926f149-ef3d-4d31-a1fb-de1912c2c5d1-sys\") pod \"perf-node-gather-daemonset-b7txh\" (UID: \"f926f149-ef3d-4d31-a1fb-de1912c2c5d1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:33.874932 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.874719    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f926f149-ef3d-4d31-a1fb-de1912c2c5d1-proc\") pod \"perf-node-gather-daemonset-b7txh\" (UID: \"f926f149-ef3d-4d31-a1fb-de1912c2c5d1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:33.874932 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.874778    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f926f149-ef3d-4d31-a1fb-de1912c2c5d1-lib-modules\") pod \"perf-node-gather-daemonset-b7txh\" (UID: \"f926f149-ef3d-4d31-a1fb-de1912c2c5d1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:33.874932 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.874787    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f926f149-ef3d-4d31-a1fb-de1912c2c5d1-sys\") pod \"perf-node-gather-daemonset-b7txh\" (UID: \"f926f149-ef3d-4d31-a1fb-de1912c2c5d1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:33.874932 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.874812    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpd2f\" (UniqueName: \"kubernetes.io/projected/f926f149-ef3d-4d31-a1fb-de1912c2c5d1-kube-api-access-zpd2f\") pod \"perf-node-gather-daemonset-b7txh\" (UID: \"f926f149-ef3d-4d31-a1fb-de1912c2c5d1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:33.874932 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.874895    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f926f149-ef3d-4d31-a1fb-de1912c2c5d1-lib-modules\") pod \"perf-node-gather-daemonset-b7txh\" (UID: \"f926f149-ef3d-4d31-a1fb-de1912c2c5d1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:33.874932 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.874928    4521 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f926f149-ef3d-4d31-a1fb-de1912c2c5d1-podres\") pod \"perf-node-gather-daemonset-b7txh\" (UID: \"f926f149-ef3d-4d31-a1fb-de1912c2c5d1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:33.875234 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.874934    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f926f149-ef3d-4d31-a1fb-de1912c2c5d1-proc\") pod \"perf-node-gather-daemonset-b7txh\" (UID: \"f926f149-ef3d-4d31-a1fb-de1912c2c5d1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:33.875234 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.875002    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f926f149-ef3d-4d31-a1fb-de1912c2c5d1-podres\") pod \"perf-node-gather-daemonset-b7txh\" (UID: \"f926f149-ef3d-4d31-a1fb-de1912c2c5d1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:33.937553 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:33.937515    4521 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpd2f\" (UniqueName: \"kubernetes.io/projected/f926f149-ef3d-4d31-a1fb-de1912c2c5d1-kube-api-access-zpd2f\") pod \"perf-node-gather-daemonset-b7txh\" (UID: \"f926f149-ef3d-4d31-a1fb-de1912c2c5d1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:34.041978 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:34.041943    4521 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:34.417961 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:34.417933    4521 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"]
Nov 04 03:52:34.420180 ip-10-0-135-242 kubenswrapper[4521]: W1104 03:52:34.420147    4521 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf926f149_ef3d_4d31_a1fb_de1912c2c5d1.slice/crio-f53441f56fcf4730f6988d0407c3cd63bf570b8efe646471ce831c540f747256 WatchSource:0}: Error finding container f53441f56fcf4730f6988d0407c3cd63bf570b8efe646471ce831c540f747256: Status 404 returned error can't find the container with id f53441f56fcf4730f6988d0407c3cd63bf570b8efe646471ce831c540f747256
Nov 04 03:52:34.673007 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:34.672924    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-hb9vr_bd98323d-8102-4f46-8765-c7ee20226eb7/kuadrant-console-plugin/0.log"
Nov 04 03:52:35.358986 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:35.358947    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh" event={"ID":"f926f149-ef3d-4d31-a1fb-de1912c2c5d1","Type":"ContainerStarted","Data":"ff0e31c28fc1c69e1df6acaeea3309f2f5ee64cc5559ea168ecb4b7e6e4386cc"}
Nov 04 03:52:35.359381 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:35.359010    4521 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh" event={"ID":"f926f149-ef3d-4d31-a1fb-de1912c2c5d1","Type":"ContainerStarted","Data":"f53441f56fcf4730f6988d0407c3cd63bf570b8efe646471ce831c540f747256"}
Nov 04 03:52:35.359381 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:35.359036    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:35.769212 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:35.769188    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7cd9b8cccb-7bml5_8af2ef47-8b2a-4065-ad7f-09e70bed85e7/manager/0.log"
Nov 04 03:52:37.580219 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:37.580180    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-n2wm8_9754c3f0-c1b7-431d-844e-0536a6654972/manager/0.log"
Nov 04 03:52:39.070597 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:39.070569    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fcfc254d-52a4-4beb-8772-d2ba0ba39cd1/alertmanager/0.log"
Nov 04 03:52:39.510167 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:39.510090    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fcfc254d-52a4-4beb-8772-d2ba0ba39cd1/config-reloader/0.log"
Nov 04 03:52:39.951928 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:39.951884    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fcfc254d-52a4-4beb-8772-d2ba0ba39cd1/kube-rbac-proxy-web/0.log"
Nov 04 03:52:40.392042 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:40.392017    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fcfc254d-52a4-4beb-8772-d2ba0ba39cd1/kube-rbac-proxy/0.log"
Nov 04 03:52:40.829309 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:40.829282    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fcfc254d-52a4-4beb-8772-d2ba0ba39cd1/kube-rbac-proxy-metric/0.log"
Nov 04 03:52:41.293934 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:41.293900    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fcfc254d-52a4-4beb-8772-d2ba0ba39cd1/prom-label-proxy/0.log"
Nov 04 03:52:41.374329 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:41.374300    4521 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh"
Nov 04 03:52:41.493144 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:41.493091    4521 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-b7txh" podStartSLOduration=8.493076777 podStartE2EDuration="8.493076777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:52:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:52:35.477024282 +0000 UTC m=+2121.681624583" watchObservedRunningTime="2026-11-04 03:52:41.493076777 +0000 UTC m=+2127.697677079"
Nov 04 03:52:41.859505 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:41.859475    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fcfc254d-52a4-4beb-8772-d2ba0ba39cd1/init-config-reloader/0.log"
Nov 04 03:52:43.027276 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:43.027246    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7659c7d778-65h9l_da5d1d62-7e67-423b-95ac-8c809a64959c/kube-auth-proxy/0.log"
Nov 04 03:52:43.710356 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:43.710331    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-dh692_ba2f1b97-4d8d-4b69-bf55-531600544660/istio-proxy/0.log"
Nov 04 03:52:44.083028 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:44.082922    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7697dbf688-n4qqr_000af384-a1dc-49b1-a49c-46e1c8a3d030/metrics-server/0.log"
Nov 04 03:52:44.160467 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:44.160433    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-fms7t_4a66091b-11c6-4dcb-8870-0ededd2187d6/istio-proxy/0.log"
Nov 04 03:52:45.065895 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:45.065818    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-sr9mv_a71c9284-671b-4ed4-8aac-7687a2e3eefe/istio-proxy/0.log"
Nov 04 03:52:45.293172 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:45.293136    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-st9jq_beb6c720-f8d2-4337-8faa-c9e5e6414d33/istio-proxy/0.log"
Nov 04 03:52:45.873550 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:45.873510    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-chld2_d770e48e-5587-4da6-838b-f7b52d18c86a/payload-pre-processing/0.log"
Nov 04 03:52:46.098711 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:46.098685    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-khhp6_cabd37d3-8208-4220-80c4-2d28ed781ce3/payload-processing/0.log"
Nov 04 03:52:47.690432 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:47.690401    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xng65_b3f8b142-e183-45b1-a9b8-1581a885b0c0/node-exporter/0.log"
Nov 04 03:52:48.135946 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:48.135916    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xng65_b3f8b142-e183-45b1-a9b8-1581a885b0c0/kube-rbac-proxy/0.log"
Nov 04 03:52:48.574119 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:48.574091    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xng65_b3f8b142-e183-45b1-a9b8-1581a885b0c0/init-textfile/0.log"
Nov 04 03:52:50.991134 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:50.991108    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-hb9vr_bd98323d-8102-4f46-8765-c7ee20226eb7/kuadrant-console-plugin/0.log"
Nov 04 03:52:53.327714 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:53.327682    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-67vg7_de7361e1-276a-489b-bfb8-140e94f3dd57/prometheus-operator/0.log"
Nov 04 03:52:53.773237 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:53.773211    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-67vg7_de7361e1-276a-489b-bfb8-140e94f3dd57/kube-rbac-proxy/0.log"
Nov 04 03:52:54.222307 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:54.222230    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-hw6vw_50350caf-69d6-47f8-9e21-c19439ce983e/prometheus-operator-admission-webhook/0.log"
Nov 04 03:52:54.670295 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:54.670262    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8b8f5df56-mxb7w_9c4dce54-5199-40ce-abfd-dae5daab91ef/telemeter-client/0.log"
Nov 04 03:52:55.113001 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:55.112965    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8b8f5df56-mxb7w_9c4dce54-5199-40ce-abfd-dae5daab91ef/reload/0.log"
Nov 04 03:52:55.551101 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:55.551048    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8b8f5df56-mxb7w_9c4dce54-5199-40ce-abfd-dae5daab91ef/kube-rbac-proxy/0.log"
Nov 04 03:52:55.999337 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:55.999261    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b89bc6d87-b96wk_c55446fd-0548-421c-8460-4b03026dfaa8/thanos-query/0.log"
Nov 04 03:52:56.474181 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:56.474149    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b89bc6d87-b96wk_c55446fd-0548-421c-8460-4b03026dfaa8/kube-rbac-proxy-web/0.log"
Nov 04 03:52:56.913440 ip-10-0-135-242 kubenswrapper[4521]: I1104 03:52:56.913408    4521 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b89bc6d87-b96wk_c55446fd-0548-421c-8460-4b03026dfaa8/kube-rbac-proxy/0.log"