Nov 02 01:31:10.122425 ip-10-0-138-75 systemd[1]: Starting Kubernetes Kubelet...
Nov 02 01:31:10.482690 ip-10-0-138-75 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 01:31:10.482690 ip-10-0-138-75 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 02 01:31:10.482690 ip-10-0-138-75 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 01:31:10.482690 ip-10-0-138-75 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 01:31:10.489414 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489314    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 01:31:10.489414 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489408    2563 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 01:31:10.489414 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489414    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 01:31:10.489414 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489418    2563 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 01:31:10.489414 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489422    2563 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489427    2563 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489431    2563 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489435    2563 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489438    2563 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489442    2563 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489446    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489449    2563 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489455    2563 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489462    2563 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489466    2563 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489471    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489476    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489480    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489486    2563 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489490    2563 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489495    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489499    2563 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489503    2563 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 01:31:10.489736 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489507    2563 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489510    2563 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489514    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489518    2563 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489522    2563 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489526    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489530    2563 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489534    2563 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489545    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489549    2563 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489553    2563 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489558    2563 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489562    2563 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489567    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489572    2563 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489577    2563 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489581    2563 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489585    2563 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489590    2563 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489594    2563 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 01:31:10.490505 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489598    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489602    2563 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489607    2563 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489611    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489616    2563 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489620    2563 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489624    2563 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489630    2563 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489635    2563 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489639    2563 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489643    2563 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489647    2563 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489651    2563 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489656    2563 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489660    2563 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489664    2563 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489668    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489672    2563 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489676    2563 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489681    2563 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 01:31:10.491324 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489685    2563 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489690    2563 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489694    2563 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489708    2563 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489713    2563 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489718    2563 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489722    2563 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489727    2563 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489731    2563 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489736    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489740    2563 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489743    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489747    2563 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489751    2563 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489756    2563 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489760    2563 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489765    2563 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489769    2563 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489774    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 01:31:10.492157 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489779    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489784    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489788    2563 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489792    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489796    2563 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489801    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489805    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489810    2563 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489814    2563 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489819    2563 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489823    2563 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489828    2563 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489832    2563 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489837    2563 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489841    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489845    2563 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489849    2563 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489853    2563 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489857    2563 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 01:31:10.493005 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489861    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489866    2563 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489871    2563 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489876    2563 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.489880    2563 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491207    2563 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491217    2563 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491222    2563 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491226    2563 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491230    2563 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491235    2563 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491239    2563 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491244    2563 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491248    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491252    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491259    2563 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491263    2563 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491268    2563 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491273    2563 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491278    2563 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 01:31:10.493620 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491282    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491286    2563 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491291    2563 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491295    2563 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491299    2563 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491303    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491308    2563 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491312    2563 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491316    2563 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491320    2563 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491324    2563 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491328    2563 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491332    2563 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491337    2563 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491341    2563 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491345    2563 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491349    2563 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491353    2563 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491358    2563 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491363    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 01:31:10.494237 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491367    2563 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491372    2563 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491376    2563 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491380    2563 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491384    2563 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491389    2563 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491393    2563 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491397    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491402    2563 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491407    2563 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491412    2563 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491416    2563 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491420    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491425    2563 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491429    2563 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491433    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491437    2563 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491441    2563 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491446    2563 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491451    2563 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 01:31:10.495074 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491455    2563 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491460    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491465    2563 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491470    2563 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491476    2563 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491484    2563 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491489    2563 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491493    2563 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491498    2563 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491502    2563 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491506    2563 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491510    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491514    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491518    2563 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491522    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491527    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491531    2563 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491535    2563 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491539    2563 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 01:31:10.495943 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491543    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491548    2563 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491553    2563 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491558    2563 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491562    2563 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491566    2563 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491571    2563 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491575    2563 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491579    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491583    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491587    2563 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491592    2563 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491596    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491600    2563 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491605    2563 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491609    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491614    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491619    2563 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491623    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 01:31:10.496617 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491627    2563 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491631    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491636    2563 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491640    2563 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491644    2563 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491648    2563 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491653    2563 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491657    2563 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491663    2563 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491669    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491676    2563 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491681    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.491686    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492525    2563 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492534    2563 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492540    2563 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492545    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492550    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492554    2563 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 01:31:10.497153 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492559    2563 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492563    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492568    2563 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492572    2563 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492577    2563 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492581    2563 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492586    2563 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492590    2563 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492594    2563 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492598    2563 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492603    2563 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492607    2563 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492614    2563 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492621    2563 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492626    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492630    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492636    2563 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492641    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492648    2563 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 01:31:10.497783 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492653    2563 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492658    2563 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492662    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492666    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492670    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492675    2563 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492679    2563 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492683    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492687    2563 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492692    2563 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492700    2563 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492704    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492709    2563 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492713    2563 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492717    2563 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492721    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492726    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492730    2563 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492735    2563 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 01:31:10.498260 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492739    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492743    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492747    2563 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492751    2563 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492756    2563 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492760    2563 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492764    2563 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492768    2563 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492773    2563 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492778    2563 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492782    2563 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492786    2563 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492790    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492794    2563 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492799    2563 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492803    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492807    2563 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492811    2563 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492816    2563 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492820    2563 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 01:31:10.498760 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492824    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492828    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492832    2563 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492838    2563 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492843    2563 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492847    2563 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492852    2563 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492856    2563 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492863    2563 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492867    2563 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492871    2563 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492875    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492879    2563 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492884    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492888    2563 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492892    2563 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492897    2563 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492901    2563 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492905    2563 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492909    2563 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 01:31:10.499284 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492913    2563 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492917    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492921    2563 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492925    2563 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492929    2563 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492933    2563 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492938    2563 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492943    2563 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492947    2563 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492951    2563 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492955    2563 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492959    2563 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492963    2563 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492967    2563 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492971    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492977    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492984    2563 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492989    2563 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492993    2563 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 01:31:10.499778 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.492997    2563 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.493003    2563 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.493007    2563 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.493983    2563 flags.go:64] FLAG: --address="0.0.0.0"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494001    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494011    2563 flags.go:64] FLAG: --anonymous-auth="true"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494018    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494026    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494031    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494039    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494045    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494050    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494055    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494062    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494067    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494072    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494077    2563 flags.go:64] FLAG: --cgroup-root=""
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494082    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494086    2563 flags.go:64] FLAG: --client-ca-file=""
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494091    2563 flags.go:64] FLAG: --cloud-provider="external"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494096    2563 flags.go:64] FLAG: --cluster-dns="[]"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494105    2563 flags.go:64] FLAG: --cluster-domain=""
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494109    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494131    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 02 01:31:10.500248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494136    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494142    2563 flags.go:64] FLAG: --container-log-max-files="5"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494149    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494153    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494159    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494164    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494169    2563 flags.go:64] FLAG: --contention-profiling="false"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494175    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494180    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494190    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494195    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494202    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494207    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494213    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494217    2563 flags.go:64] FLAG: --enable-load-reader="false"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494222    2563 flags.go:64] FLAG: --enable-server="true"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494227    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494233    2563 flags.go:64] FLAG: --event-burst="100"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494238    2563 flags.go:64] FLAG: --event-qps="50"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494242    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494247    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494251    2563 flags.go:64] FLAG: --eviction-hard=""
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494258    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494262    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494268    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 02 01:31:10.500797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494274    2563 flags.go:64] FLAG: --eviction-soft=""
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494279    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494284    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494289    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494293    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494298    2563 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494303    2563 flags.go:64] FLAG: --fail-swap-on="true"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494308    2563 flags.go:64] FLAG: --feature-gates=""
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494315    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494320    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494325    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494331    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494336    2563 flags.go:64] FLAG: --healthz-port="10248"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494340    2563 flags.go:64] FLAG: --help="false"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494345    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494353    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494359    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494364    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494369    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494375    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494380    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494384    2563 flags.go:64] FLAG: --image-service-endpoint=""
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494389    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 02 01:31:10.501401 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494394    2563 flags.go:64] FLAG: --kube-api-burst="100"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494399    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494404    2563 flags.go:64] FLAG: --kube-api-qps="50"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494409    2563 flags.go:64] FLAG: --kube-reserved=""
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494414    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494418    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494423    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494428    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494433    2563 flags.go:64] FLAG: --lock-file=""
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494438    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494443    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494448    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494457    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494462    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494467    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494471    2563 flags.go:64] FLAG: --logging-format="text"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494476    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494481    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494486    2563 flags.go:64] FLAG: --manifest-url=""
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494491    2563 flags.go:64] FLAG: --manifest-url-header=""
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494499    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494503    2563 flags.go:64] FLAG: --max-open-files="1000000"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494509    2563 flags.go:64] FLAG: --max-pods="110"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494514    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494519    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 02 01:31:10.501967 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494526    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494531    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494536    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494541    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494546    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494557    2563 flags.go:64] FLAG: --node-status-max-images="50"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494562    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494567    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494572    2563 flags.go:64] FLAG: --pod-cidr=""
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494576    2563 flags.go:64] FLAG: --pod-manifest-path=""
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494581    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494586    2563 flags.go:64] FLAG: --pods-per-core="0"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494590    2563 flags.go:64] FLAG: --port="10250"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494595    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494600    2563 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-069eb0397cf7993ea"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494605    2563 flags.go:64] FLAG: --qos-reserved=""
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494610    2563 flags.go:64] FLAG: --read-only-port="10255"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494614    2563 flags.go:64] FLAG: --register-node="true"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494619    2563 flags.go:64] FLAG: --register-with-taints=""
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494625    2563 flags.go:64] FLAG: --registry-burst="10"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494630    2563 flags.go:64] FLAG: --registry-qps="5"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494634    2563 flags.go:64] FLAG: --reserved-cpus=""
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494639    2563 flags.go:64] FLAG: --reserved-memory=""
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494646    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494651    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494656    2563 flags.go:64] FLAG: --rotate-certificates="false"
Nov 02 01:31:10.502570 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494660    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494665    2563 flags.go:64] FLAG: --runonce="false"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494670    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494675    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494679    2563 flags.go:64] FLAG: --seccomp-default="false"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494684    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494689    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494695    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494700    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494706    2563 flags.go:64] FLAG: --storage-driver-password="root"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494710    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494715    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494719    2563 flags.go:64] FLAG: --storage-driver-user="root"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494724    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494729    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494734    2563 flags.go:64] FLAG: --system-cgroups=""
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494739    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494747    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494755    2563 flags.go:64] FLAG: --tls-cert-file=""
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494760    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494768    2563 flags.go:64] FLAG: --tls-min-version=""
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494772    2563 flags.go:64] FLAG: --tls-private-key-file=""
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494777    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494782    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494787    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 02 01:31:10.503196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494792    2563 flags.go:64] FLAG: --v="2"
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494798    2563 flags.go:64] FLAG: --version="false"
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494805    2563 flags.go:64] FLAG: --vmodule=""
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494811    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.494817    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495036    2563 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495043    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495048    2563 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495052    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495057    2563 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495061    2563 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495066    2563 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495070    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495074    2563 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495078    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495084    2563 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495091    2563 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495098    2563 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495103    2563 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495109    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 01:31:10.503797 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495129    2563 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495135    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495139    2563 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495143    2563 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495147    2563 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495154    2563 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495158    2563 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495162    2563 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495166    2563 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495170    2563 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495175    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495179    2563 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495184    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495188    2563 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495192    2563 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495196    2563 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495201    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495205    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495209    2563 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 01:31:10.504292 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495213    2563 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495218    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495222    2563 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495227    2563 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495231    2563 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495235    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495241    2563 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495245    2563 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495250    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495265    2563 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495270    2563 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495274    2563 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495279    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495283    2563 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495287    2563 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495291    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495298    2563 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495304    2563 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495310    2563 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 01:31:10.504720 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495315    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495319    2563 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495324    2563 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495328    2563 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495333    2563 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495338    2563 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495343    2563 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495347    2563 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495351    2563 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495356    2563 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495360    2563 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495364    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495368    2563 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495372    2563 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495377    2563 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495382    2563 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495386    2563 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495391    2563 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495395    2563 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495401    2563 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 01:31:10.505187 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495405    2563 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495410    2563 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495414    2563 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495420    2563 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495424    2563 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495429    2563 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495433    2563 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495437    2563 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495442    2563 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495446    2563 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495450    2563 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495457    2563 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495462    2563 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495466    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495471    2563 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495475    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495479    2563 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495484    2563 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495488    2563 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495492    2563 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 01:31:10.505721 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495496    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 01:31:10.506204 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495501    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 01:31:10.506204 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495505    2563 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 01:31:10.506204 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495509    2563 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 01:31:10.506204 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495514    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 01:31:10.506204 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495518    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 01:31:10.506204 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495523    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 01:31:10.506204 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495527    2563 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 01:31:10.506204 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495531    2563 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 01:31:10.506204 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495535    2563 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 01:31:10.506204 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495540    2563 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 01:31:10.506204 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495546    2563 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 01:31:10.506204 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495551    2563 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 01:31:10.506204 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.495595    2563 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 01:31:10.506204 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495679    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 01:31:10.506204 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495686    2563 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 01:31:10.506204 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495692    2563 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495697    2563 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495702    2563 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495706    2563 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495710    2563 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495715    2563 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495719    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495723    2563 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495729    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495733    2563 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495737    2563 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495742    2563 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495746    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495750    2563 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495755    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495759    2563 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495766    2563 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495772    2563 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495777    2563 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 01:31:10.506584 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495781    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495785    2563 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495790    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495794    2563 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495799    2563 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495803    2563 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495807    2563 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495811    2563 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495817    2563 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495821    2563 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495826    2563 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495830    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495834    2563 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495839    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495844    2563 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495848    2563 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495852    2563 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495856    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495860    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 01:31:10.507027 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495865    2563 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495869    2563 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495875    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495879    2563 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495884    2563 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495888    2563 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495892    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495897    2563 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495901    2563 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495905    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495909    2563 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495914    2563 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495918    2563 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495922    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495926    2563 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495931    2563 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495935    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495942    2563 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495948    2563 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 01:31:10.507483 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495952    2563 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495957    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495963    2563 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495967    2563 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495971    2563 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495976    2563 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495980    2563 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495984    2563 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495989    2563 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495994    2563 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.495998    2563 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496002    2563 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496007    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496011    2563 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496015    2563 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496021    2563 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496025    2563 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496030    2563 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496034    2563 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496038    2563 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 01:31:10.507919 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496042    2563 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496047    2563 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496051    2563 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496055    2563 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496059    2563 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496063    2563 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496067    2563 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496072    2563 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496076    2563 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496080    2563 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496085    2563 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496089    2563 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496093    2563 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496097    2563 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496103    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496107    2563 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496111    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496131    2563 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496135    2563 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496140    2563 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 01:31:10.508412 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496144    2563 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496148    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496153    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496157    2563 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496161    2563 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496167    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.496171    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.503189    2563 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.503207    2563 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.503224    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.503229    2563 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503287    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503292    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503295    2563 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503298    2563 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503302    2563 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503305    2563 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503307    2563 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503310    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 01:31:10.508872 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503313    2563 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503315    2563 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503318    2563 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503321    2563 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503324    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503326    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503329    2563 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503332    2563 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503334    2563 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503337    2563 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503339    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503342    2563 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503345    2563 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503347    2563 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503350    2563 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503354    2563 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503358    2563 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503361    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503364    2563 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 01:31:10.509451 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503367    2563 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503369    2563 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503374    2563 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503377    2563 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503379    2563 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503383    2563 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503385    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503388    2563 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503390    2563 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503393    2563 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503395    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503398    2563 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503401    2563 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503403    2563 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503406    2563 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503408    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503411    2563 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503414    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503416    2563 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 01:31:10.509899 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503419    2563 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503421    2563 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503424    2563 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503426    2563 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503429    2563 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503431    2563 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503434    2563 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503436    2563 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503439    2563 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503441    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503444    2563 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503446    2563 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503449    2563 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503451    2563 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503454    2563 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503457    2563 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503461    2563 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503463    2563 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503466    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503469    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 01:31:10.510359 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503471    2563 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503474    2563 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503476    2563 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503479    2563 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503481    2563 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503484    2563 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503486    2563 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503489    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503491    2563 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503494    2563 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503496    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503499    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503501    2563 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503504    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503506    2563 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503509    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503511    2563 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503514    2563 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503516    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503519    2563 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 01:31:10.511044 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503521    2563 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503524    2563 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503526    2563 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503529    2563 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503532    2563 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503535    2563 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503538    2563 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503540    2563 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503543    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503546    2563 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503548    2563 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503551    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503554    2563 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503556    2563 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503559    2563 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503561    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503564    2563 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503566    2563 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503569    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 01:31:10.511651 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503573    2563 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 01:31:10.512135 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.503600    2563 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 01:31:10.512135 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503757    2563 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 01:31:10.512135 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503763    2563 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 01:31:10.512135 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503768    2563 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 01:31:10.512135 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503771    2563 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 01:31:10.512135 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503774    2563 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 01:31:10.512135 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503788    2563 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 01:31:10.512135 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503792    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 01:31:10.512135 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503795    2563 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 01:31:10.512135 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503798    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 01:31:10.512135 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503801    2563 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 01:31:10.512135 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503804    2563 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 01:31:10.512135 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503807    2563 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 01:31:10.512135 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503809    2563 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 01:31:10.512135 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503812    2563 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503814    2563 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503818    2563 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503821    2563 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503823    2563 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503826    2563 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503828    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503831    2563 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503834    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503837    2563 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503840    2563 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503842    2563 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503845    2563 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503847    2563 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503850    2563 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503852    2563 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503855    2563 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503858    2563 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503860    2563 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 01:31:10.512756 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503863    2563 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503866    2563 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503870    2563 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503872    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503875    2563 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503877    2563 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503880    2563 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503883    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503885    2563 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503887    2563 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503890    2563 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503893    2563 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503895    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503898    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503900    2563 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503903    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503906    2563 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503908    2563 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503911    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 01:31:10.513340 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503913    2563 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503916    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503918    2563 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503921    2563 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503924    2563 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503926    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503929    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503931    2563 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503934    2563 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503936    2563 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503939    2563 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503941    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503944    2563 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503946    2563 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503949    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503951    2563 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503954    2563 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503956    2563 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503959    2563 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503961    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 01:31:10.513793 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503964    2563 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503966    2563 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503969    2563 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503971    2563 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503974    2563 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503977    2563 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503979    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503982    2563 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503985    2563 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503987    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503990    2563 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503992    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503995    2563 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.503997    2563 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504000    2563 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504002    2563 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504005    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504008    2563 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504010    2563 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504013    2563 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 01:31:10.514579 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504015    2563 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 01:31:10.515261 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504018    2563 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 01:31:10.515261 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504020    2563 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 01:31:10.515261 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504022    2563 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 01:31:10.515261 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504025    2563 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 01:31:10.515261 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504027    2563 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 01:31:10.515261 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504030    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 01:31:10.515261 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504032    2563 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 01:31:10.515261 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504035    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 01:31:10.515261 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504037    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 01:31:10.515261 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504040    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 01:31:10.515261 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504042    2563 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 01:31:10.515261 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504045    2563 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 01:31:10.515261 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504047    2563 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 01:31:10.515261 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504050    2563 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 01:31:10.515261 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.504074    2563 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 01:31:10.515261 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504107    2563 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504111    2563 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504132    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504136    2563 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504138    2563 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504141    2563 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504144    2563 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504146    2563 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504149    2563 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504151    2563 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504154    2563 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504156    2563 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504159    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504162    2563 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504165    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504168    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504170    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504172    2563 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504175    2563 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 01:31:10.515630 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504177    2563 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504180    2563 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504182    2563 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504186    2563 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504190    2563 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504192    2563 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504195    2563 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504197    2563 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504200    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504203    2563 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504208    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504210    2563 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504213    2563 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504215    2563 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504218    2563 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504220    2563 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504222    2563 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504225    2563 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 01:31:10.516096 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504228    2563 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504230    2563 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504233    2563 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504235    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504238    2563 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504240    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504243    2563 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504245    2563 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504248    2563 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504251    2563 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504254    2563 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504256    2563 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504259    2563 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504261    2563 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504264    2563 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504266    2563 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504269    2563 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504272    2563 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504276    2563 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504278    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 01:31:10.516525 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504282    2563 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504284    2563 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504287    2563 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504290    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504293    2563 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504296    2563 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504299    2563 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504302    2563 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504304    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504307    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504309    2563 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504312    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504314    2563 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504317    2563 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504320    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504322    2563 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504325    2563 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504327    2563 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504330    2563 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504332    2563 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 01:31:10.517002 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504334    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504337    2563 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504344    2563 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504347    2563 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504350    2563 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504352    2563 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504355    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504357    2563 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504360    2563 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504363    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504365    2563 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504368    2563 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504370    2563 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504373    2563 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504375    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504378    2563 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504383    2563 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504386    2563 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504388    2563 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504391    2563 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 01:31:10.517491 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504393    2563 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 01:31:10.518211 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504395    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 01:31:10.518211 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504398    2563 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 01:31:10.518211 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504400    2563 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 01:31:10.518211 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504403    2563 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 01:31:10.518211 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504405    2563 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 01:31:10.518211 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504408    2563 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 01:31:10.518211 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504410    2563 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 01:31:10.518211 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:10.504414    2563 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 01:31:10.518211 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.504468    2563 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 02 01:31:10.518211 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.506795    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 02 01:31:10.518211 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.507733    2563 server.go:1014] "Starting client certificate rotation"
Nov 02 01:31:10.518211 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.507831    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 01:31:10.518211 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.507872    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 01:31:10.518211 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.509813    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 01:31:10.518740 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.518671    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 01:31:10.530366 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.530352    2563 log.go:25] "Validated CRI v1 runtime API"
Nov 02 01:31:10.534091 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.534076    2563 log.go:25] "Validated CRI v1 image API"
Nov 02 01:31:10.534864 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.534838    2563 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 02 01:31:10.537195 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.537177    2563 fs.go:135] Filesystem UUIDs: map[4fb30ead-4668-44ea-8407-816ebc85e31c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a2df9fa6-7dd1-4f64-b662-7bf7cd9195b5:/dev/nvme0n1p3]
Nov 02 01:31:10.537257 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.537195    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 02 01:31:10.542091 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.541982    2563 manager.go:217] Machine: {Timestamp:2026-11-02 01:31:10.540767477 +0000 UTC m=+0.321873715 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099475 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d2cf2c1e7f59217e112280a3e32e0 SystemUUID:ec2d2cf2-c1e7-f592-17e1-12280a3e32e0 BootID:4db775b0-7797-4307-ba92-ee34e4d47977 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:56:ca:d2:fc:b5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:56:ca:d2:fc:b5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:a1:a2:fd:bf:00 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 02 01:31:10.542189 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.542092    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 02 01:31:10.542189 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.542179    2563 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 02 01:31:10.543305 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.543280    2563 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 02 01:31:10.543426 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.543305    2563 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-75.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 02 01:31:10.543473 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.543429    2563 topology_manager.go:143] "Creating topology manager with none policy"
Nov 02 01:31:10.543473 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.543437    2563 container_manager_linux.go:311] "Creating device plugin manager"
Nov 02 01:31:10.543473 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.543471    2563 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 01:31:10.543574 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.543489    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 01:31:10.544525 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.544516    2563 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 02 01:31:10.547910 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.547898    2563 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 02 01:31:10.548024 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.548016    2563 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 02 01:31:10.548765 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.548755    2563 kubelet.go:493] "Attempting to sync node with API server"
Nov 02 01:31:10.548804 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.548771    2563 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 02 01:31:10.548804 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.548782    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 02 01:31:10.548804 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.548792    2563 kubelet.go:405] "Adding apiserver pod source"
Nov 02 01:31:10.548804 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.548799    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 02 01:31:10.549129 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.549105    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 01:31:10.551407 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.551390    2563 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 02 01:31:10.559400 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.559384    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 01:31:10.560257 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.560245    2563 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 02 01:31:10.560299 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.560275    2563 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 02 01:31:10.560426 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.560416    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 02 01:31:10.560455 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.560432    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 02 01:31:10.560455 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.560439    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 02 01:31:10.560455 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.560445    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 02 01:31:10.560455 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.560451    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 02 01:31:10.560550 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.560457    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 02 01:31:10.561268 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.561259    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 02 01:31:10.561299 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.561269    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 02 01:31:10.561299 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.561277    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 02 01:31:10.561299 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.561283    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 02 01:31:10.561299 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.561292    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 02 01:31:10.561299 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.561300    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 02 01:31:10.562980 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.562971    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 02 01:31:10.562980 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.562981    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 02 01:31:10.566626 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.566615    2563 server.go:1263] "Started kubelet"
Nov 02 01:31:10.566779 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.566723    2563 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 02 01:31:10.566779 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.566707    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 02 01:31:10.566884 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.566811    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 02 01:31:10.567359 ip-10-0-138-75 systemd[1]: Started Kubernetes Kubelet.
Nov 02 01:31:10.568385 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.568369    2563 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 02 01:31:10.569613 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.569597    2563 server.go:317] "Adding debug handlers to kubelet server"
Nov 02 01:31:10.571977 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:10.571957    2563 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 02 01:31:10.575518 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.575501    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 02 01:31:10.576225 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.576210    2563 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 02 01:31:10.576722 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:10.576322    2563 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-75.us-west-1.compute.internal\" not found"
Nov 02 01:31:10.577040 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.577024    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 02 01:31:10.577040 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.577025    2563 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 02 01:31:10.577190 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.577044    2563 factory.go:55] Registering systemd factory
Nov 02 01:31:10.577190 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.577057    2563 factory.go:223] Registration of the systemd container factory successfully
Nov 02 01:31:10.577190 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.577049    2563 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 02 01:31:10.577190 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.577186    2563 reconstruct.go:97] "Volume reconstruction finished"
Nov 02 01:31:10.577190 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.577192    2563 reconciler.go:29] "Reconciler: start to sync state"
Nov 02 01:31:10.577425 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.577297    2563 factory.go:153] Registering CRI-O factory
Nov 02 01:31:10.577425 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.577311    2563 factory.go:223] Registration of the crio container factory successfully
Nov 02 01:31:10.577425 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.577364    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 02 01:31:10.577425 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.577381    2563 factory.go:103] Registering Raw factory
Nov 02 01:31:10.577425 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.577393    2563 manager.go:1199] Started watching for new ooms in manager
Nov 02 01:31:10.578463 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.578443    2563 manager.go:319] Starting recovery of all containers
Nov 02 01:31:10.591100 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.590387    2563 manager.go:324] Recovery completed
Nov 02 01:31:10.596704 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.596686    2563 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 01:31:10.598686 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.598667    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 01:31:10.598764 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.598705    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 01:31:10.598764 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.598719    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 01:31:10.599316 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.599300    2563 cpu_manager.go:226] "Starting" policy="none"
Nov 02 01:31:10.599316 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.599315    2563 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 02 01:31:10.599460 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.599335    2563 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 02 01:31:10.599460 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.599317    2563 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 01:31:10.600177 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.600111    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 01:31:10.600258 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.600185    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 01:31:10.600258 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.600195    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 01:31:10.600349 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.600298    2563 policy_none.go:50] "Start"
Nov 02 01:31:10.600349 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.600311    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 02 01:31:10.600349 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.600325    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 02 01:31:10.601326 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.601313    2563 policy_none.go:44] "Start"
Nov 02 01:31:10.630737 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.630688    2563 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 02 01:31:10.634686 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.634665    2563 manager.go:341] "Starting Device Plugin manager"
Nov 02 01:31:10.634768 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:10.634697    2563 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 02 01:31:10.634768 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.634706    2563 server.go:87] "Starting device plugin registration server"
Nov 02 01:31:10.634942 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.634928    2563 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 02 01:31:10.634987 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.634944    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 02 01:31:10.635053 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.635040    2563 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 02 01:31:10.635152 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.635137    2563 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 02 01:31:10.635152 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.635149    2563 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 02 01:31:10.635707 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:10.635690    2563 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 02 01:31:10.635796 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:10.635742    2563 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-75.us-west-1.compute.internal\" not found"
Nov 02 01:31:10.742709 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.705066    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 02 01:31:10.742709 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.706368    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 02 01:31:10.742709 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.706392    2563 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 02 01:31:10.742709 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.706415    2563 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 02 01:31:10.742709 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:10.706453    2563 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 02 01:31:10.742709 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.708226    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-75.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 02 01:31:10.742709 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:10.708341    2563 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-75.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 02 01:31:10.742709 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:10.708242    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-75.us-west-1.compute.internal.18e4164cc446a2c6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-75.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-75.us-west-1.compute.internal,},FirstTimestamp:2026-11-02 01:31:10.566589126 +0000 UTC m=+0.347695362,LastTimestamp:2026-11-02 01:31:10.566589126 +0000 UTC m=+0.347695362,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-75.us-west-1.compute.internal,}"
Nov 02 01:31:10.742709 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.735148    2563 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 01:31:10.742709 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.735996    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 01:31:10.742709 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.736026    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 01:31:10.742709 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.736037    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 01:31:10.742709 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.736059    2563 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:10.791087 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.791062    2563 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 02 01:31:10.791231 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.791102    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lpq52"
Nov 02 01:31:10.807146 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.807126    2563 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal"]
Nov 02 01:31:10.807194 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.807178    2563 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 01:31:10.808036 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.808019    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 01:31:10.808093 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.808050    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 01:31:10.808093 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.808060    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 01:31:10.809020 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.809009    2563 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 01:31:10.809204 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.809190    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:10.809248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.809224    2563 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 01:31:10.809803 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.809781    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 01:31:10.809891 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.809814    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 01:31:10.809891 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.809822    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 01:31:10.809891 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.809846    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 01:31:10.809891 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.809858    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 01:31:10.809891 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.809829    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 01:31:10.810907 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.810894    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:10.810955 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.810923    2563 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 01:31:10.811553 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.811539    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 01:31:10.811627 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.811566    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 01:31:10.811627 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.811588    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 01:31:10.839601 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:10.839584    2563 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-75.us-west-1.compute.internal\" not found" node="ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:10.843430 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:10.843413    2563 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-75.us-west-1.compute.internal\" not found" node="ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:10.867603 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.867584    2563 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 01:31:10.868803 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.868788    2563 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 01:31:10.966336 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.966311    2563 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 01:31:10.967689 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.967672    2563 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 01:31:10.967749 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.967706    2563 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:10.967749 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:10.967728    2563 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-75.us-west-1.compute.internal\": node \"ip-10-0-138-75.us-west-1.compute.internal\" not found"
Nov 02 01:31:10.977048 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.977028    2563 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:10.978184 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.978169    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe959ce17f01eeb1eaeed7edd9cd6e8b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"fe959ce17f01eeb1eaeed7edd9cd6e8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:10.978237 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.978192    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e1f480d0264a553be3287c8f045c192e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"e1f480d0264a553be3287c8f045c192e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:10.978237 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:10.978211    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe959ce17f01eeb1eaeed7edd9cd6e8b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"fe959ce17f01eeb1eaeed7edd9cd6e8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:11.037398 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.037348    2563 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 01:31:11.037477 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.037437    2563 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:11.078904 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.078885    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe959ce17f01eeb1eaeed7edd9cd6e8b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"fe959ce17f01eeb1eaeed7edd9cd6e8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:11.078950 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.078910    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe959ce17f01eeb1eaeed7edd9cd6e8b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"fe959ce17f01eeb1eaeed7edd9cd6e8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:11.078950 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.078927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e1f480d0264a553be3287c8f045c192e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"e1f480d0264a553be3287c8f045c192e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:11.079021 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.078979    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e1f480d0264a553be3287c8f045c192e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"e1f480d0264a553be3287c8f045c192e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:11.079216 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.078975    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe959ce17f01eeb1eaeed7edd9cd6e8b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"fe959ce17f01eeb1eaeed7edd9cd6e8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:11.079318 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.079295    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe959ce17f01eeb1eaeed7edd9cd6e8b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"fe959ce17f01eeb1eaeed7edd9cd6e8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:11.092602 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.092584    2563 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 01:31:11.142988 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.142967    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:11.145883 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.145859    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:11.507963 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.507934    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 02 01:31:11.508524 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.508073    2563 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 02 01:31:11.508524 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.508111    2563 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 02 01:31:11.508524 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:11.508189    2563 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a86811a309f80445ca4534cfcca7e946-0363337bf0817064.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-138-75.us-west-1.compute.internal.18e4164cc630fd32\": read tcp 10.0.138.75:58524->3.142.175.121:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-138-75.us-west-1.compute.internal.18e4164cc630fd32  default   4957 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-75.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-75.us-west-1.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-75.us-west-1.compute.internal,},FirstTimestamp:2026-11-02 01:31:10 +0000 UTC,LastTimestamp:2026-11-02 01:31:10.736041042 +0000 UTC m=+0.517147279,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-75.us-west-1.compute.internal,}"
Nov 02 01:31:11.549407 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.549351    2563 apiserver.go:52] "Watching apiserver"
Nov 02 01:31:11.576664 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.576636    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 02 01:31:11.793926 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.793887    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-01 01:26:10 +0000 UTC" deadline="2028-05-09 16:50:46.130114804 +0000 UTC"
Nov 02 01:31:11.793926 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.793920    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13311h19m34.336198922s"
Nov 02 01:31:11.827581 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.827512    2563 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 02 01:31:11.827840 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.827821    2563 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-nth82","openshift-network-operator/iptables-alerter-9bccs","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal","openshift-ovn-kubernetes/ovnkube-node-gs99s","kube-system/global-pull-secret-syncer-cqspb","openshift-image-registry/node-ca-z5pbt","openshift-multus/multus-f6pjf","kube-system/konnectivity-agent-hbcss","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52","openshift-cluster-node-tuning-operator/tuned-47cxd","openshift-dns/node-resolver-cfv7t","openshift-multus/multus-additional-cni-plugins-9k54v","openshift-multus/network-metrics-daemon-p75zz","kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal"]
Nov 02 01:31:11.829089 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.829072    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.830125 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.830100    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:11.831879 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.831860    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hbcss"
Nov 02 01:31:11.832849 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.832832    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.834022 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.834001    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:11.834112 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:11.834085    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nth82" podUID="ed8d1d91-e435-42f1-a875-f87fd18e0b87"
Nov 02 01:31:11.835147 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.835113    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:11.836232 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.836217    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.837242 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.837225    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:11.837322 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:11.837304    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p75zz" podUID="1092321a-6edf-43d5-bb31-5efb0b80990e"
Nov 02 01:31:11.838218 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.838205    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:11.839452 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.839436    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.840590 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.840576    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:11.840651 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:11.840623    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cqspb" podUID="696e3b37-d3bb-48e4-9836-afb85f9e34fd"
Nov 02 01:31:11.841573 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.841551    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.880815 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.879276    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 02 01:31:11.881663 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.881640    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-ovn\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.881753 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.881677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-cni-bin\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.881753 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.881703    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cni-binary-copy\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.881845 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.881768    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0aafee8b-ab11-4b97-afae-09b9a6d59c01-host-slash\") pod \"iptables-alerter-9bccs\" (UID: \"0aafee8b-ab11-4b97-afae-09b9a6d59c01\") " pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:11.881845 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.881816    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-tuned\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.881925 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.881842    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-tmp\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.881925 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.881867    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-multus-certs\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.881925 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.881890    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-cni-netd\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.882026 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.881937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-env-overrides\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.882026 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.881963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovn-node-metrics-cert\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.882026 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.881994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-socket-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.882026 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882020    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-device-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.882183 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882045    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.882183 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882072    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdkwl\" (UniqueName: \"kubernetes.io/projected/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-kube-api-access-mdkwl\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.882183 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-modprobe-d\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.882183 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-cni-binary-copy\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.882183 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882168    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfc7g\" (UniqueName: \"kubernetes.io/projected/0aafee8b-ab11-4b97-afae-09b9a6d59c01-kube-api-access-dfc7g\") pod \"iptables-alerter-9bccs\" (UID: \"0aafee8b-ab11-4b97-afae-09b9a6d59c01\") " pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:11.882366 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882192    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovnkube-script-lib\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.882366 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cnibin\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.882366 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882301    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.882366 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.882366 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882353    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-node-log\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.882580 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882378    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-cni-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.882580 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882402    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-daemon-config\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.882580 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882463    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-etc-kubernetes\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.882580 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882495    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skmsl\" (UniqueName: \"kubernetes.io/projected/1092321a-6edf-43d5-bb31-5efb0b80990e-kube-api-access-skmsl\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:11.882580 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882530    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysconfig\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.882580 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-var-lib-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.882849 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882581    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-cnibin\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.882849 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882603    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-netns\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.882849 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882627    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-cni-bin\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.882849 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882654    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-run-netns\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.882849 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882678    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-run-ovn-kubernetes\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.882849 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882704    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjz92\" (UniqueName: \"kubernetes.io/projected/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-kube-api-access-pjz92\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.882849 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882728    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/50bfe6af-769e-40aa-8a09-560ea1a984a2-serviceca\") pod \"node-ca-z5pbt\" (UID: \"50bfe6af-769e-40aa-8a09-560ea1a984a2\") " pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:11.882849 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882751    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsdhq\" (UniqueName: \"kubernetes.io/projected/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-kube-api-access-vsdhq\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.882849 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882777    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0aafee8b-ab11-4b97-afae-09b9a6d59c01-iptables-alerter-script\") pod \"iptables-alerter-9bccs\" (UID: \"0aafee8b-ab11-4b97-afae-09b9a6d59c01\") " pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:11.882849 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882811    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-kubernetes\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.882849 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882849    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysctl-conf\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.883235 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882885    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-run\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.883235 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882906    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-sys\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.883235 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882921    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovnkube-config\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.883235 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.882974    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/696e3b37-d3bb-48e4-9836-afb85f9e34fd-kubelet-config\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:11.883235 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-system-cni-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.883235 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883015    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0923847d-508c-4308-93f9-543d9bb3c251-agent-certs\") pod \"konnectivity-agent-hbcss\" (UID: \"0923847d-508c-4308-93f9-543d9bb3c251\") " pod="kube-system/konnectivity-agent-hbcss"
Nov 02 01:31:11.883235 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rws7g\" (UniqueName: \"kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g\") pod \"network-check-target-nth82\" (UID: \"ed8d1d91-e435-42f1-a875-f87fd18e0b87\") " pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:11.883235 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883071    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrjqz\" (UniqueName: \"kubernetes.io/projected/50bfe6af-769e-40aa-8a09-560ea1a984a2-kube-api-access-jrjqz\") pod \"node-ca-z5pbt\" (UID: \"50bfe6af-769e-40aa-8a09-560ea1a984a2\") " pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:11.883235 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/696e3b37-d3bb-48e4-9836-afb85f9e34fd-dbus\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:11.883235 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883100    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:11.883235 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883124    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99m65\" (UniqueName: \"kubernetes.io/projected/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-kube-api-access-99m65\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.883235 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883181    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-cni-multus\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.883235 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-systemd-units\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883262    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50bfe6af-769e-40aa-8a09-560ea1a984a2-host\") pod \"node-ca-z5pbt\" (UID: \"50bfe6af-769e-40aa-8a09-560ea1a984a2\") " pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883320    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-k8s-cni-cncf-io\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883384    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kctw\" (UniqueName: \"kubernetes.io/projected/414870a7-ef47-4b1e-aac2-1b25923286e9-kube-api-access-2kctw\") pod \"node-resolver-cfv7t\" (UID: \"414870a7-ef47-4b1e-aac2-1b25923286e9\") " pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883408    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0923847d-508c-4308-93f9-543d9bb3c251-konnectivity-ca\") pod \"konnectivity-agent-hbcss\" (UID: \"0923847d-508c-4308-93f9-543d9bb3c251\") " pod="kube-system/konnectivity-agent-hbcss"
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883431    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysctl-d\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883455    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-sys-fs\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883528    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-vt6x2\""
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883545    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-socket-dir-parent\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883516    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883580    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-lib-modules\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883615    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883627    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjsm4\" (UniqueName: \"kubernetes.io/projected/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-kube-api-access-gjsm4\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883662    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-slash\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883708    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883745    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-log-socket\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883749    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883747    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-z6qvz\""
Nov 02 01:31:11.883805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883770    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-etc-selinux\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.884555 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883798    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-system-cni-dir\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.884555 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883819    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-systemd\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.884555 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-var-lib-kubelet\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.884555 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883924    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-kubelet\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.884555 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883949    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-kubelet\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.884555 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883968    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/414870a7-ef47-4b1e-aac2-1b25923286e9-hosts-file\") pod \"node-resolver-cfv7t\" (UID: \"414870a7-ef47-4b1e-aac2-1b25923286e9\") " pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:11.884555 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883982    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-systemd\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.884555 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.883997    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.884555 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.884042    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-hostroot\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.884555 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.884081    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.884555 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.884147    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-registration-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.884555 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.884178    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-conf-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.884555 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.884203    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/414870a7-ef47-4b1e-aac2-1b25923286e9-tmp-dir\") pod \"node-resolver-cfv7t\" (UID: \"414870a7-ef47-4b1e-aac2-1b25923286e9\") " pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:11.884555 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.884228    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:11.884555 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.884251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-host\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.884555 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.884274    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-etc-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.885563 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.885542    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-kk5lr\""
Nov 02 01:31:11.885563 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.885546    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 02 01:31:11.885706 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.885652    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 02 01:31:11.886685 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.886540    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 02 01:31:11.886685 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.886542    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 02 01:31:11.886685 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.886670    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 02 01:31:11.886800 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.886785    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 02 01:31:11.886901 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.886886    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xbzw4\""
Nov 02 01:31:11.886948 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.886926    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 02 01:31:11.886980 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.886958    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 02 01:31:11.888801 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.888785    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 02 01:31:11.888912 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.888860    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 02 01:31:11.932983 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.932965    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 02 01:31:11.933112 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.933095    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-n6fsn\""
Nov 02 01:31:11.935126 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.935111    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 02 01:31:11.935186 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.935158    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 02 01:31:11.935334 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.935322    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 02 01:31:11.935367 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.935339    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 02 01:31:11.935410 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.935396    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 02 01:31:11.935444 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.935408    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xv9qq\""
Nov 02 01:31:11.935444 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.935403    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 02 01:31:11.937058 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.937042    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 02 01:31:11.937058 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.937052    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-87db6\""
Nov 02 01:31:11.937140 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.937059    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 02 01:31:11.937140 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.937054    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j52qq\""
Nov 02 01:31:11.937246 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.937235    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 02 01:31:11.937296 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.937283    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 02 01:31:11.938185 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.938170    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wfswr\""
Nov 02 01:31:11.938252 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.938239    2563 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 02 01:31:11.938305 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.938292    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bvgdw"
Nov 02 01:31:11.985378 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985358    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjsm4\" (UniqueName: \"kubernetes.io/projected/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-kube-api-access-gjsm4\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.985459 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985395    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-slash\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.985459 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985420    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-log-socket\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.985459 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985449    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-etc-selinux\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.985578 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985474    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-system-cni-dir\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.985578 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985477    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-slash\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.985578 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985501    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-systemd\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.985578 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985548    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-system-cni-dir\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.985578 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985570    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-systemd\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.985578 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985573    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-log-socket\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-var-lib-kubelet\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985602    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-etc-selinux\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985629    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-kubelet\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985635    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-var-lib-kubelet\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985650    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-kubelet\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-kubelet\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/414870a7-ef47-4b1e-aac2-1b25923286e9-hosts-file\") pod \"node-resolver-cfv7t\" (UID: \"414870a7-ef47-4b1e-aac2-1b25923286e9\") " pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-systemd\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985722    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-kubelet\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985730    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/414870a7-ef47-4b1e-aac2-1b25923286e9-hosts-file\") pod \"node-resolver-cfv7t\" (UID: \"414870a7-ef47-4b1e-aac2-1b25923286e9\") " pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985737    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985760    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-systemd\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-hostroot\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985786    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985811    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-registration-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985844    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-conf-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985868    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-hostroot\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.985878 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985885    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.986672 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985890    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-registration-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.986672 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985870    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/414870a7-ef47-4b1e-aac2-1b25923286e9-tmp-dir\") pod \"node-resolver-cfv7t\" (UID: \"414870a7-ef47-4b1e-aac2-1b25923286e9\") " pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:11.986672 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985912    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-conf-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.986672 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:11.986672 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985955    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-host\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.986672 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.985978    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-etc-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.986672 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986003    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-ovn\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.986672 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986039    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-host\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.986672 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986045    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-etc-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.986672 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:11.986049    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 01:31:11.986672 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986046    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-cni-bin\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.986672 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986082    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cni-binary-copy\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.986672 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986088    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-ovn\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.986672 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986094    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-cni-bin\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.986672 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:11.986115    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs podName:1092321a-6edf-43d5-bb31-5efb0b80990e nodeName:}" failed. No retries permitted until 2026-11-02 01:31:12.486095483 +0000 UTC m=+2.267201731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs") pod "network-metrics-daemon-p75zz" (UID: "1092321a-6edf-43d5-bb31-5efb0b80990e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 01:31:11.986672 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986152    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0aafee8b-ab11-4b97-afae-09b9a6d59c01-host-slash\") pod \"iptables-alerter-9bccs\" (UID: \"0aafee8b-ab11-4b97-afae-09b9a6d59c01\") " pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:11.986672 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986177    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0aafee8b-ab11-4b97-afae-09b9a6d59c01-host-slash\") pod \"iptables-alerter-9bccs\" (UID: \"0aafee8b-ab11-4b97-afae-09b9a6d59c01\") " pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-tuned\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986180    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/414870a7-ef47-4b1e-aac2-1b25923286e9-tmp-dir\") pod \"node-resolver-cfv7t\" (UID: \"414870a7-ef47-4b1e-aac2-1b25923286e9\") " pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986207    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-tmp\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986257    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-multus-certs\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986284    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-cni-netd\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-env-overrides\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovn-node-metrics-cert\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986338    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-multus-certs\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986369    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-socket-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986411    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-device-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986441    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdkwl\" (UniqueName: \"kubernetes.io/projected/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-kube-api-access-mdkwl\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986502    2563 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986514    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-socket-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986529    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986597    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-cni-netd\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986510    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-modprobe-d\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.987475 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986610    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-modprobe-d\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.988303 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-cni-binary-copy\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.988303 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986652    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-device-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.988303 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986658    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfc7g\" (UniqueName: \"kubernetes.io/projected/0aafee8b-ab11-4b97-afae-09b9a6d59c01-kube-api-access-dfc7g\") pod \"iptables-alerter-9bccs\" (UID: \"0aafee8b-ab11-4b97-afae-09b9a6d59c01\") " pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:11.988303 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986684    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovnkube-script-lib\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.988303 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986709    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cnibin\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.988303 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986734    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.988303 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.988303 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986783    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-node-log\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.988303 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986798    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-cni-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.988303 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-daemon-config\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.988303 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986874    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-etc-kubernetes\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.988303 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skmsl\" (UniqueName: \"kubernetes.io/projected/1092321a-6edf-43d5-bb31-5efb0b80990e-kube-api-access-skmsl\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:11.988303 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986922    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysconfig\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.988303 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986971    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-env-overrides\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.988303 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.986974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-var-lib-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.988303 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987029    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-cnibin\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.988303 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987044    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cni-binary-copy\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987055    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-netns\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987079    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-netns\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987100    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-cni-bin\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987118    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-cnibin\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987139    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-run-netns\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987148    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-cni-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-run-ovn-kubernetes\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987190    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cnibin\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987197    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjz92\" (UniqueName: \"kubernetes.io/projected/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-kube-api-access-pjz92\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987364    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987399    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/50bfe6af-769e-40aa-8a09-560ea1a984a2-serviceca\") pod \"node-ca-z5pbt\" (UID: \"50bfe6af-769e-40aa-8a09-560ea1a984a2\") " pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987402    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-cni-binary-copy\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987428    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsdhq\" (UniqueName: \"kubernetes.io/projected/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-kube-api-access-vsdhq\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987434    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-cni-bin\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987465    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-etc-kubernetes\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987469    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-run-netns\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987454    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0aafee8b-ab11-4b97-afae-09b9a6d59c01-iptables-alerter-script\") pod \"iptables-alerter-9bccs\" (UID: \"0aafee8b-ab11-4b97-afae-09b9a6d59c01\") " pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:11.988978 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987503    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-run-ovn-kubernetes\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987514    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovnkube-script-lib\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987520    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-kubernetes\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987547    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysctl-conf\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987569    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-var-lib-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987573    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-run\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987599    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987607    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-run\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987603    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-sys\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987628    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-sys\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987650    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovnkube-config\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987660    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987671    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysconfig\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987688    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/696e3b37-d3bb-48e4-9836-afb85f9e34fd-kubelet-config\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987704    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-node-log\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987713    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-system-cni-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987739    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0923847d-508c-4308-93f9-543d9bb3c251-agent-certs\") pod \"konnectivity-agent-hbcss\" (UID: \"0923847d-508c-4308-93f9-543d9bb3c251\") " pod="kube-system/konnectivity-agent-hbcss"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987763    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rws7g\" (UniqueName: \"kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g\") pod \"network-check-target-nth82\" (UID: \"ed8d1d91-e435-42f1-a875-f87fd18e0b87\") " pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:11.989825 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987787    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrjqz\" (UniqueName: \"kubernetes.io/projected/50bfe6af-769e-40aa-8a09-560ea1a984a2-kube-api-access-jrjqz\") pod \"node-ca-z5pbt\" (UID: \"50bfe6af-769e-40aa-8a09-560ea1a984a2\") " pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:11.990656 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/696e3b37-d3bb-48e4-9836-afb85f9e34fd-dbus\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:11.990656 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987835    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:11.990656 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987862    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99m65\" (UniqueName: \"kubernetes.io/projected/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-kube-api-access-99m65\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:11.990656 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-cni-multus\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.990656 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987944    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-systemd-units\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.990656 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987970    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50bfe6af-769e-40aa-8a09-560ea1a984a2-host\") pod \"node-ca-z5pbt\" (UID: \"50bfe6af-769e-40aa-8a09-560ea1a984a2\") " pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:11.990656 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987995    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-k8s-cni-cncf-io\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.990656 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.988021    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kctw\" (UniqueName: \"kubernetes.io/projected/414870a7-ef47-4b1e-aac2-1b25923286e9-kube-api-access-2kctw\") pod \"node-resolver-cfv7t\" (UID: \"414870a7-ef47-4b1e-aac2-1b25923286e9\") " pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:11.990656 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.988046    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0923847d-508c-4308-93f9-543d9bb3c251-konnectivity-ca\") pod \"konnectivity-agent-hbcss\" (UID: \"0923847d-508c-4308-93f9-543d9bb3c251\") " pod="kube-system/konnectivity-agent-hbcss"
Nov 02 01:31:11.990656 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.988071    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysctl-d\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.990656 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.988099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.990656 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.988109    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/696e3b37-d3bb-48e4-9836-afb85f9e34fd-kubelet-config\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:11.990656 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.988151    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/696e3b37-d3bb-48e4-9836-afb85f9e34fd-dbus\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:11.990656 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.988178    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-sys-fs\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.990656 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:11.988234    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 01:31:11.990656 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.988238    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-kubernetes\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.990656 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.988278    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50bfe6af-769e-40aa-8a09-560ea1a984a2-host\") pod \"node-ca-z5pbt\" (UID: \"50bfe6af-769e-40aa-8a09-560ea1a984a2\") " pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:11.991395 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:11.988304    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret podName:696e3b37-d3bb-48e4-9836-afb85f9e34fd nodeName:}" failed. No retries permitted until 2026-11-02 01:31:12.488267137 +0000 UTC m=+2.269373378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret") pod "global-pull-secret-syncer-cqspb" (UID: "696e3b37-d3bb-48e4-9836-afb85f9e34fd") : object "kube-system"/"original-pull-secret" not registered
Nov 02 01:31:11.991395 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.988741    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-k8s-cni-cncf-io\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.991395 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.988825    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/50bfe6af-769e-40aa-8a09-560ea1a984a2-serviceca\") pod \"node-ca-z5pbt\" (UID: \"50bfe6af-769e-40aa-8a09-560ea1a984a2\") " pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:11.991395 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.988927    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0aafee8b-ab11-4b97-afae-09b9a6d59c01-iptables-alerter-script\") pod \"iptables-alerter-9bccs\" (UID: \"0aafee8b-ab11-4b97-afae-09b9a6d59c01\") " pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:11.991395 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.988975    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovnkube-config\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.991395 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.989029    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-cni-multus\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.991395 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.988116    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-sys-fs\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:11.991395 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.989485    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-socket-dir-parent\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.991395 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.987915    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-daemon-config\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.991395 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.989529    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-lib-modules\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.991395 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.989624    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-systemd-units\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.991395 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.989678    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-socket-dir-parent\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.991395 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.989755    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-lib-modules\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.991395 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.989759    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-system-cni-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:11.991395 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.989867    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysctl-d\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.991395 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.989893    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysctl-conf\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.991395 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.989926    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.992136 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.990639    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0923847d-508c-4308-93f9-543d9bb3c251-konnectivity-ca\") pod \"konnectivity-agent-hbcss\" (UID: \"0923847d-508c-4308-93f9-543d9bb3c251\") " pod="kube-system/konnectivity-agent-hbcss"
Nov 02 01:31:11.992136 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.990716    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovn-node-metrics-cert\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:11.994342 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.994317    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-tuned\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.994498 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.994476    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-tmp\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:11.994580 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:11.994480    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0923847d-508c-4308-93f9-543d9bb3c251-agent-certs\") pod \"konnectivity-agent-hbcss\" (UID: \"0923847d-508c-4308-93f9-543d9bb3c251\") " pod="kube-system/konnectivity-agent-hbcss"
Nov 02 01:31:12.048402 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.048367    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjsm4\" (UniqueName: \"kubernetes.io/projected/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-kube-api-access-gjsm4\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:12.049248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.049220    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfc7g\" (UniqueName: \"kubernetes.io/projected/0aafee8b-ab11-4b97-afae-09b9a6d59c01-kube-api-access-dfc7g\") pod \"iptables-alerter-9bccs\" (UID: \"0aafee8b-ab11-4b97-afae-09b9a6d59c01\") " pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:12.049248 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.049237    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdkwl\" (UniqueName: \"kubernetes.io/projected/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-kube-api-access-mdkwl\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:12.049611 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.049594    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjz92\" (UniqueName: \"kubernetes.io/projected/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-kube-api-access-pjz92\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:12.053772 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:12.053045    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 01:31:12.053772 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:12.053065    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 01:31:12.053772 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:12.053078    2563 projected.go:196] Error preparing data for projected volume kube-api-access-rws7g for pod openshift-network-diagnostics/network-check-target-nth82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 01:31:12.053772 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:12.053150    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g podName:ed8d1d91-e435-42f1-a875-f87fd18e0b87 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:12.553132716 +0000 UTC m=+2.334238957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rws7g" (UniqueName: "kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g") pod "network-check-target-nth82" (UID: "ed8d1d91-e435-42f1-a875-f87fd18e0b87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 01:31:12.054483 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.054412    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrjqz\" (UniqueName: \"kubernetes.io/projected/50bfe6af-769e-40aa-8a09-560ea1a984a2-kube-api-access-jrjqz\") pod \"node-ca-z5pbt\" (UID: \"50bfe6af-769e-40aa-8a09-560ea1a984a2\") " pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:12.055708 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.055686    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kctw\" (UniqueName: \"kubernetes.io/projected/414870a7-ef47-4b1e-aac2-1b25923286e9-kube-api-access-2kctw\") pod \"node-resolver-cfv7t\" (UID: \"414870a7-ef47-4b1e-aac2-1b25923286e9\") " pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:12.056419 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.056401    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99m65\" (UniqueName: \"kubernetes.io/projected/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-kube-api-access-99m65\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:12.056520 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.056455    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsdhq\" (UniqueName: \"kubernetes.io/projected/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-kube-api-access-vsdhq\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:12.056657 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.056633    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skmsl\" (UniqueName: \"kubernetes.io/projected/1092321a-6edf-43d5-bb31-5efb0b80990e-kube-api-access-skmsl\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:12.167288 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.167219    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:12.187339 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.187314    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:12.202323 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.202303    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hbcss"
Nov 02 01:31:12.220977 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.220953    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:12.228527 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.228504    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:12.241275 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:12.241246    2563 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f7a5eb_5a3c_4bd5_a0cd_486539e66c82.slice/crio-982ddaade8fa1cf87d9a02fbaff8ccf6d1c21e5e30027a5ded433d0ec1fc1c24 WatchSource:0}: Error finding container 982ddaade8fa1cf87d9a02fbaff8ccf6d1c21e5e30027a5ded433d0ec1fc1c24: Status 404 returned error can't find the container with id 982ddaade8fa1cf87d9a02fbaff8ccf6d1c21e5e30027a5ded433d0ec1fc1c24
Nov 02 01:31:12.241855 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:12.241833    2563 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f480d0264a553be3287c8f045c192e.slice/crio-d1e73f09594b5e38baccb49b8c5d3d2936ef28ad507f41fa1d8db246627b9a16 WatchSource:0}: Error finding container d1e73f09594b5e38baccb49b8c5d3d2936ef28ad507f41fa1d8db246627b9a16: Status 404 returned error can't find the container with id d1e73f09594b5e38baccb49b8c5d3d2936ef28ad507f41fa1d8db246627b9a16
Nov 02 01:31:12.243251 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:12.243230    2563 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50bfe6af_769e_40aa_8a09_560ea1a984a2.slice/crio-819ee76eaddc5e64334fc9b9cf78fd4e98a5f24bb9338793fe78562a091f87e3 WatchSource:0}: Error finding container 819ee76eaddc5e64334fc9b9cf78fd4e98a5f24bb9338793fe78562a091f87e3: Status 404 returned error can't find the container with id 819ee76eaddc5e64334fc9b9cf78fd4e98a5f24bb9338793fe78562a091f87e3
Nov 02 01:31:12.243966 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:12.243947    2563 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe959ce17f01eeb1eaeed7edd9cd6e8b.slice/crio-e896a197f81fdba8c0f0e3a2b999f02eecbcfc614349e74b00e3c79f2fec99aa WatchSource:0}: Error finding container e896a197f81fdba8c0f0e3a2b999f02eecbcfc614349e74b00e3c79f2fec99aa: Status 404 returned error can't find the container with id e896a197f81fdba8c0f0e3a2b999f02eecbcfc614349e74b00e3c79f2fec99aa
Nov 02 01:31:12.247804 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.247788    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 01:31:12.250349 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.250327    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:12.258010 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:12.257986    2563 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a3c8cf1_c66c_4f81_9a40_c19a26df3f77.slice/crio-b6a21ea7b12c785fcee3884c1c9e1539ba4d90fca5283b36f93bebd60ba0ca38 WatchSource:0}: Error finding container b6a21ea7b12c785fcee3884c1c9e1539ba4d90fca5283b36f93bebd60ba0ca38: Status 404 returned error can't find the container with id b6a21ea7b12c785fcee3884c1c9e1539ba4d90fca5283b36f93bebd60ba0ca38
Nov 02 01:31:12.268220 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.268204    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:12.275287 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:12.275269    2563 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aafee8b_ab11_4b97_afae_09b9a6d59c01.slice/crio-ecb4de11d1b780ab7bdc06ace820e4fae200e440969d25b6b1e2f1e1c402d672 WatchSource:0}: Error finding container ecb4de11d1b780ab7bdc06ace820e4fae200e440969d25b6b1e2f1e1c402d672: Status 404 returned error can't find the container with id ecb4de11d1b780ab7bdc06ace820e4fae200e440969d25b6b1e2f1e1c402d672
Nov 02 01:31:12.279294 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.279281    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:12.286035 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:12.286017    2563 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode33b967e_3f0d_49c0_bae2_c95ce8341e1f.slice/crio-bc3a7fa2b613ed811c2e2ef23351a83a60b71c673e421c44bf587e7739391711 WatchSource:0}: Error finding container bc3a7fa2b613ed811c2e2ef23351a83a60b71c673e421c44bf587e7739391711: Status 404 returned error can't find the container with id bc3a7fa2b613ed811c2e2ef23351a83a60b71c673e421c44bf587e7739391711
Nov 02 01:31:12.295913 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.295895    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:12.302702 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:12.302685    2563 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf40cfbd7_701b_4fb2_8756_86ae988cb1b8.slice/crio-98e4014dc2542786d0aa2a7ccace60f106061391d004e725a0ce667b45cdfbab WatchSource:0}: Error finding container 98e4014dc2542786d0aa2a7ccace60f106061391d004e725a0ce667b45cdfbab: Status 404 returned error can't find the container with id 98e4014dc2542786d0aa2a7ccace60f106061391d004e725a0ce667b45cdfbab
Nov 02 01:31:12.492557 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.492529    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:12.492713 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.492593    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:12.492713 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:12.492671    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 01:31:12.492713 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:12.492678    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 01:31:12.492823 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:12.492724    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs podName:1092321a-6edf-43d5-bb31-5efb0b80990e nodeName:}" failed. No retries permitted until 2026-11-02 01:31:13.492707502 +0000 UTC m=+3.273813744 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs") pod "network-metrics-daemon-p75zz" (UID: "1092321a-6edf-43d5-bb31-5efb0b80990e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 01:31:12.492823 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:12.492738    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret podName:696e3b37-d3bb-48e4-9836-afb85f9e34fd nodeName:}" failed. No retries permitted until 2026-11-02 01:31:13.49273155 +0000 UTC m=+3.273837777 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret") pod "global-pull-secret-syncer-cqspb" (UID: "696e3b37-d3bb-48e4-9836-afb85f9e34fd") : object "kube-system"/"original-pull-secret" not registered
Nov 02 01:31:12.593177 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.593150    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rws7g\" (UniqueName: \"kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g\") pod \"network-check-target-nth82\" (UID: \"ed8d1d91-e435-42f1-a875-f87fd18e0b87\") " pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:12.593793 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:12.593296    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 01:31:12.593793 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:12.593311    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 01:31:12.593793 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:12.593320    2563 projected.go:196] Error preparing data for projected volume kube-api-access-rws7g for pod openshift-network-diagnostics/network-check-target-nth82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 01:31:12.593793 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:12.593362    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g podName:ed8d1d91-e435-42f1-a875-f87fd18e0b87 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:13.593350779 +0000 UTC m=+3.374457004 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rws7g" (UniqueName: "kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g") pod "network-check-target-nth82" (UID: "ed8d1d91-e435-42f1-a875-f87fd18e0b87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 01:31:12.710533 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.710483    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal" event={"ID":"e1f480d0264a553be3287c8f045c192e","Type":"ContainerStarted","Data":"d1e73f09594b5e38baccb49b8c5d3d2936ef28ad507f41fa1d8db246627b9a16"}
Nov 02 01:31:12.711519 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.711494    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal" event={"ID":"fe959ce17f01eeb1eaeed7edd9cd6e8b","Type":"ContainerStarted","Data":"e896a197f81fdba8c0f0e3a2b999f02eecbcfc614349e74b00e3c79f2fec99aa"}
Nov 02 01:31:12.712516 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.712500    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9k54v" event={"ID":"f40cfbd7-701b-4fb2-8756-86ae988cb1b8","Type":"ContainerStarted","Data":"98e4014dc2542786d0aa2a7ccace60f106061391d004e725a0ce667b45cdfbab"}
Nov 02 01:31:12.713523 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.713507    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-47cxd" event={"ID":"e33b967e-3f0d-49c0-bae2-c95ce8341e1f","Type":"ContainerStarted","Data":"bc3a7fa2b613ed811c2e2ef23351a83a60b71c673e421c44bf587e7739391711"}
Nov 02 01:31:12.714535 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.714518    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9bccs" event={"ID":"0aafee8b-ab11-4b97-afae-09b9a6d59c01","Type":"ContainerStarted","Data":"ecb4de11d1b780ab7bdc06ace820e4fae200e440969d25b6b1e2f1e1c402d672"}
Nov 02 01:31:12.715524 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.715508    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52" event={"ID":"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77","Type":"ContainerStarted","Data":"b6a21ea7b12c785fcee3884c1c9e1539ba4d90fca5283b36f93bebd60ba0ca38"}
Nov 02 01:31:12.716522 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.716505    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z5pbt" event={"ID":"50bfe6af-769e-40aa-8a09-560ea1a984a2","Type":"ContainerStarted","Data":"819ee76eaddc5e64334fc9b9cf78fd4e98a5f24bb9338793fe78562a091f87e3"}
Nov 02 01:31:12.717518 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.717499    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s" event={"ID":"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82","Type":"ContainerStarted","Data":"982ddaade8fa1cf87d9a02fbaff8ccf6d1c21e5e30027a5ded433d0ec1fc1c24"}
Nov 02 01:31:12.860460 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.860389    2563 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 01:31:12.907885 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.907857    2563 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 01:31:12.939877 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.939846    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 01:26:11 +0000 UTC" deadline="2028-04-26 12:38:09.011478787 +0000 UTC"
Nov 02 01:31:12.939877 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:12.939873    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12995h6m56.071609432s"
Nov 02 01:31:12.962023 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:12.961997    2563 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0d7fa8f_7d4f_4ef5_a35d_1f3a132cc9db.slice/crio-ded9e93ebf8ca912e18a1a7f8bfd8ec8d008e1fe8cca740687bc5b3ecc4af381 WatchSource:0}: Error finding container ded9e93ebf8ca912e18a1a7f8bfd8ec8d008e1fe8cca740687bc5b3ecc4af381: Status 404 returned error can't find the container with id ded9e93ebf8ca912e18a1a7f8bfd8ec8d008e1fe8cca740687bc5b3ecc4af381
Nov 02 01:31:13.037216 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:13.037184    2563 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod414870a7_ef47_4b1e_aac2_1b25923286e9.slice/crio-1a1a99bc627340193bbd40fe1a5b38f3cb0e46cdccd5ce51adea4f27cba68b5e WatchSource:0}: Error finding container 1a1a99bc627340193bbd40fe1a5b38f3cb0e46cdccd5ce51adea4f27cba68b5e: Status 404 returned error can't find the container with id 1a1a99bc627340193bbd40fe1a5b38f3cb0e46cdccd5ce51adea4f27cba68b5e
Nov 02 01:31:13.102295 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:13.102251    2563 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0923847d_508c_4308_93f9_543d9bb3c251.slice/crio-21120f85e79b6dddbb9c5ff481c813160558def131429c9f93bffbbfa01ee947 WatchSource:0}: Error finding container 21120f85e79b6dddbb9c5ff481c813160558def131429c9f93bffbbfa01ee947: Status 404 returned error can't find the container with id 21120f85e79b6dddbb9c5ff481c813160558def131429c9f93bffbbfa01ee947
Nov 02 01:31:13.501017 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:13.500976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:13.501219 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:13.501083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:13.501298 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:13.501240    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 01:31:13.501350 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:13.501312    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret podName:696e3b37-d3bb-48e4-9836-afb85f9e34fd nodeName:}" failed. No retries permitted until 2026-11-02 01:31:15.501292475 +0000 UTC m=+5.282398717 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret") pod "global-pull-secret-syncer-cqspb" (UID: "696e3b37-d3bb-48e4-9836-afb85f9e34fd") : object "kube-system"/"original-pull-secret" not registered
Nov 02 01:31:13.501350 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:13.501240    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 01:31:13.501462 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:13.501359    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs podName:1092321a-6edf-43d5-bb31-5efb0b80990e nodeName:}" failed. No retries permitted until 2026-11-02 01:31:15.50134664 +0000 UTC m=+5.282452878 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs") pod "network-metrics-daemon-p75zz" (UID: "1092321a-6edf-43d5-bb31-5efb0b80990e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 01:31:13.601903 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:13.601864    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rws7g\" (UniqueName: \"kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g\") pod \"network-check-target-nth82\" (UID: \"ed8d1d91-e435-42f1-a875-f87fd18e0b87\") " pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:13.602432 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:13.602044    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 01:31:13.602432 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:13.602063    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 01:31:13.602432 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:13.602075    2563 projected.go:196] Error preparing data for projected volume kube-api-access-rws7g for pod openshift-network-diagnostics/network-check-target-nth82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 01:31:13.602432 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:13.602148    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g podName:ed8d1d91-e435-42f1-a875-f87fd18e0b87 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:15.602129665 +0000 UTC m=+5.383235903 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rws7g" (UniqueName: "kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g") pod "network-check-target-nth82" (UID: "ed8d1d91-e435-42f1-a875-f87fd18e0b87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 01:31:13.707690 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:13.707227    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:13.707690 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:13.707361    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p75zz" podUID="1092321a-6edf-43d5-bb31-5efb0b80990e"
Nov 02 01:31:13.707690 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:13.707431    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:13.707690 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:13.707499    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nth82" podUID="ed8d1d91-e435-42f1-a875-f87fd18e0b87"
Nov 02 01:31:13.707690 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:13.707556    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:13.707690 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:13.707638    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cqspb" podUID="696e3b37-d3bb-48e4-9836-afb85f9e34fd"
Nov 02 01:31:13.734446 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:13.734408    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hbcss" event={"ID":"0923847d-508c-4308-93f9-543d9bb3c251","Type":"ContainerStarted","Data":"21120f85e79b6dddbb9c5ff481c813160558def131429c9f93bffbbfa01ee947"}
Nov 02 01:31:13.740295 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:13.740265    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cfv7t" event={"ID":"414870a7-ef47-4b1e-aac2-1b25923286e9","Type":"ContainerStarted","Data":"1a1a99bc627340193bbd40fe1a5b38f3cb0e46cdccd5ce51adea4f27cba68b5e"}
Nov 02 01:31:13.745896 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:13.745870    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f6pjf" event={"ID":"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db","Type":"ContainerStarted","Data":"ded9e93ebf8ca912e18a1a7f8bfd8ec8d008e1fe8cca740687bc5b3ecc4af381"}
Nov 02 01:31:13.940253 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:13.940198    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 01:26:11 +0000 UTC" deadline="2028-06-17 14:30:05.013308064 +0000 UTC"
Nov 02 01:31:13.940253 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:13.940251    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14244h58m51.07305898s"
Nov 02 01:31:15.517681 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:15.517646    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:15.518149 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:15.517717    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:15.518149 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:15.517847    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 01:31:15.518149 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:15.517911    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret podName:696e3b37-d3bb-48e4-9836-afb85f9e34fd nodeName:}" failed. No retries permitted until 2026-11-02 01:31:19.517893996 +0000 UTC m=+9.299000223 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret") pod "global-pull-secret-syncer-cqspb" (UID: "696e3b37-d3bb-48e4-9836-afb85f9e34fd") : object "kube-system"/"original-pull-secret" not registered
Nov 02 01:31:15.518149 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:15.517985    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 01:31:15.518149 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:15.518018    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs podName:1092321a-6edf-43d5-bb31-5efb0b80990e nodeName:}" failed. No retries permitted until 2026-11-02 01:31:19.518006595 +0000 UTC m=+9.299112821 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs") pod "network-metrics-daemon-p75zz" (UID: "1092321a-6edf-43d5-bb31-5efb0b80990e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 01:31:15.618973 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:15.618942    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rws7g\" (UniqueName: \"kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g\") pod \"network-check-target-nth82\" (UID: \"ed8d1d91-e435-42f1-a875-f87fd18e0b87\") " pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:15.619181 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:15.619164    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 01:31:15.619258 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:15.619193    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 01:31:15.619258 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:15.619206    2563 projected.go:196] Error preparing data for projected volume kube-api-access-rws7g for pod openshift-network-diagnostics/network-check-target-nth82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 01:31:15.619258 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:15.619257    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g podName:ed8d1d91-e435-42f1-a875-f87fd18e0b87 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:19.619240888 +0000 UTC m=+9.400347117 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rws7g" (UniqueName: "kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g") pod "network-check-target-nth82" (UID: "ed8d1d91-e435-42f1-a875-f87fd18e0b87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 01:31:15.707328 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:15.706875    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:15.707328 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:15.707001    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cqspb" podUID="696e3b37-d3bb-48e4-9836-afb85f9e34fd"
Nov 02 01:31:15.707328 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:15.707063    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:15.707328 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:15.707162    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p75zz" podUID="1092321a-6edf-43d5-bb31-5efb0b80990e"
Nov 02 01:31:15.707328 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:15.707207    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:15.707328 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:15.707265    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nth82" podUID="ed8d1d91-e435-42f1-a875-f87fd18e0b87"
Nov 02 01:31:16.771592 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:16.771347    2563 generic.go:356] "Generic (PLEG): container finished" podID="f40cfbd7-701b-4fb2-8756-86ae988cb1b8" containerID="6e55565b1ec11f9a13534a68679da53b4b7a28a616dcf28122e08cc05233b1db" exitCode=0
Nov 02 01:31:16.772236 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:16.771598    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9k54v" event={"ID":"f40cfbd7-701b-4fb2-8756-86ae988cb1b8","Type":"ContainerDied","Data":"6e55565b1ec11f9a13534a68679da53b4b7a28a616dcf28122e08cc05233b1db"}
Nov 02 01:31:17.707508 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:17.707481    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:17.707630 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:17.707607    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cqspb" podUID="696e3b37-d3bb-48e4-9836-afb85f9e34fd"
Nov 02 01:31:17.707700 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:17.707686    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:17.707787 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:17.707770    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p75zz" podUID="1092321a-6edf-43d5-bb31-5efb0b80990e"
Nov 02 01:31:17.707851 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:17.707828    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:17.707906 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:17.707891    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nth82" podUID="ed8d1d91-e435-42f1-a875-f87fd18e0b87"
Nov 02 01:31:17.785165 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:17.784600    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal" event={"ID":"e1f480d0264a553be3287c8f045c192e","Type":"ContainerStarted","Data":"305772659dd5f86a6224f4132efc5f940212390b77be84465ceed587b3e88b88"}
Nov 02 01:31:17.790220 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:17.789703    2563 generic.go:356] "Generic (PLEG): container finished" podID="fe959ce17f01eeb1eaeed7edd9cd6e8b" containerID="55672280fe2ed3bf86882a8dd74104d8771e770f91b578a817bf75a2a17e7d0a" exitCode=0
Nov 02 01:31:17.790220 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:17.789750    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal" event={"ID":"fe959ce17f01eeb1eaeed7edd9cd6e8b","Type":"ContainerDied","Data":"55672280fe2ed3bf86882a8dd74104d8771e770f91b578a817bf75a2a17e7d0a"}
Nov 02 01:31:17.901964 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:17.901335    2563 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal" podStartSLOduration=6.901317057 podStartE2EDuration="6.901317057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 01:31:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:31:17.900491292 +0000 UTC m=+7.681597564" watchObservedRunningTime="2026-11-02 01:31:17.901317057 +0000 UTC m=+7.682423306"
Nov 02 01:31:18.796183 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:18.796147    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z5pbt" event={"ID":"50bfe6af-769e-40aa-8a09-560ea1a984a2","Type":"ContainerStarted","Data":"27700d1fafbfcdbbe333c00c5b9a9172e5ebb6d9e1880606aefe7357424fe645"}
Nov 02 01:31:18.804380 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:18.804351    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal" event={"ID":"fe959ce17f01eeb1eaeed7edd9cd6e8b","Type":"ContainerStarted","Data":"2a17dde8335fab2e71fc4ecfbdc2ff2e747dcd16fc47c8cea17574e9d6232715"}
Nov 02 01:31:18.807300 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:18.807259    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hbcss" event={"ID":"0923847d-508c-4308-93f9-543d9bb3c251","Type":"ContainerStarted","Data":"fe71b0ee90987bbea925e8f70187b7f5f7eaea1c87c5a7c673d76200e5a00474"}
Nov 02 01:31:18.912310 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:18.912249    2563 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-z5pbt" podStartSLOduration=3.119739839 podStartE2EDuration="8.912229682s" totalImagesPullingTime="5.792489843s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 01:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:31:18.911822339 +0000 UTC m=+8.692928590" watchObservedRunningTime="2026-11-02 01:31:18.912229682 +0000 UTC m=+8.693335929"
Nov 02 01:31:19.154815 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:19.154754    2563 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal" podStartSLOduration=8.154739897 podStartE2EDuration="8.154739897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 01:31:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:31:19.035438083 +0000 UTC m=+8.816544329" watchObservedRunningTime="2026-11-02 01:31:19.154739897 +0000 UTC m=+8.935846143"
Nov 02 01:31:19.561478 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:19.561392    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:19.561478 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:19.561455    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:19.561710 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:19.561551    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 01:31:19.561710 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:19.561601    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret podName:696e3b37-d3bb-48e4-9836-afb85f9e34fd nodeName:}" failed. No retries permitted until 2026-11-02 01:31:27.561586035 +0000 UTC m=+17.342692271 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret") pod "global-pull-secret-syncer-cqspb" (UID: "696e3b37-d3bb-48e4-9836-afb85f9e34fd") : object "kube-system"/"original-pull-secret" not registered
Nov 02 01:31:19.561710 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:19.561644    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 01:31:19.561710 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:19.561665    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs podName:1092321a-6edf-43d5-bb31-5efb0b80990e nodeName:}" failed. No retries permitted until 2026-11-02 01:31:27.561657999 +0000 UTC m=+17.342764235 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs") pod "network-metrics-daemon-p75zz" (UID: "1092321a-6edf-43d5-bb31-5efb0b80990e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 01:31:19.662419 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:19.662377    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rws7g\" (UniqueName: \"kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g\") pod \"network-check-target-nth82\" (UID: \"ed8d1d91-e435-42f1-a875-f87fd18e0b87\") " pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:19.662604 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:19.662573    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 01:31:19.662604 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:19.662595    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 01:31:19.662717 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:19.662607    2563 projected.go:196] Error preparing data for projected volume kube-api-access-rws7g for pod openshift-network-diagnostics/network-check-target-nth82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 01:31:19.662717 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:19.662662    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g podName:ed8d1d91-e435-42f1-a875-f87fd18e0b87 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:27.662646286 +0000 UTC m=+17.443752525 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rws7g" (UniqueName: "kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g") pod "network-check-target-nth82" (UID: "ed8d1d91-e435-42f1-a875-f87fd18e0b87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 01:31:19.706707 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:19.706675    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:19.706880 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:19.706818    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p75zz" podUID="1092321a-6edf-43d5-bb31-5efb0b80990e"
Nov 02 01:31:19.706953 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:19.706888    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:19.707004 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:19.706957    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nth82" podUID="ed8d1d91-e435-42f1-a875-f87fd18e0b87"
Nov 02 01:31:19.707054 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:19.707011    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:19.707098 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:19.707080    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cqspb" podUID="696e3b37-d3bb-48e4-9836-afb85f9e34fd"
Nov 02 01:31:19.812343 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:19.812265    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52" event={"ID":"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77","Type":"ContainerStarted","Data":"8062e4d127077e6130d30f5313a0aef22f1aaa090d353d1ef1c93be676911f66"}
Nov 02 01:31:20.117738 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:20.117657    2563 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hbcss"
Nov 02 01:31:20.118718 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:20.118526    2563 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hbcss"
Nov 02 01:31:20.234652 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:20.234595    2563 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hbcss" podStartSLOduration=5.630937121 podStartE2EDuration="10.234577735s" totalImagesPullingTime="4.603640614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:31:19.155306041 +0000 UTC m=+8.936412297" watchObservedRunningTime="2026-11-02 01:31:20.234577735 +0000 UTC m=+10.015683980"
Nov 02 01:31:21.185096 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:21.185072    2563 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 02 01:31:21.642034 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:21.641941    2563 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-02T01:31:21.185094058Z","UUID":"39f7eca8-f81a-49b4-8544-747bbcab3712","Handler":null,"Name":"","Endpoint":""}
Nov 02 01:31:21.643509 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:21.643485    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 02 01:31:21.643622 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:21.643521    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 02 01:31:21.707285 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:21.707253    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:21.707285 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:21.707277    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:21.707493 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:21.707397    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cqspb" podUID="696e3b37-d3bb-48e4-9836-afb85f9e34fd"
Nov 02 01:31:21.707553 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:21.707508    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p75zz" podUID="1092321a-6edf-43d5-bb31-5efb0b80990e"
Nov 02 01:31:21.707622 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:21.707576    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:21.707673 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:21.707648    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nth82" podUID="ed8d1d91-e435-42f1-a875-f87fd18e0b87"
Nov 02 01:31:21.818183 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:21.818146    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52" event={"ID":"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77","Type":"ContainerStarted","Data":"0422fc30f2eeb95da318bb65d0c257eecccb919e66d9a173b79d1fed90113474"}
Nov 02 01:31:21.819748 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:21.819714    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cfv7t" event={"ID":"414870a7-ef47-4b1e-aac2-1b25923286e9","Type":"ContainerStarted","Data":"9d18c1240a56b2a4239cb1d8fab0b1719ddf6625a140ecc6d77df1c2e18f19ca"}
Nov 02 01:31:21.822091 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:21.822055    2563 generic.go:356] "Generic (PLEG): container finished" podID="f40cfbd7-701b-4fb2-8756-86ae988cb1b8" containerID="548e285af4a9dea0d4797a6d2c0312e5d80ae526e5977100547f20d7ee659642" exitCode=0
Nov 02 01:31:21.822245 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:21.822149    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9k54v" event={"ID":"f40cfbd7-701b-4fb2-8756-86ae988cb1b8","Type":"ContainerDied","Data":"548e285af4a9dea0d4797a6d2c0312e5d80ae526e5977100547f20d7ee659642"}
Nov 02 01:31:21.825370 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:21.825351    2563 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 01:31:21.825676 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:21.825658    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f6pjf" event={"ID":"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db","Type":"ContainerStarted","Data":"bcf92d3836e827c8ba8487298cfce3b57e325b826d1e9f83bdf5904cd1147349"}
Nov 02 01:31:21.933677 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:21.933415    2563 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-cfv7t" podStartSLOduration=3.692660079 podStartE2EDuration="11.933398235s" totalImagesPullingTime="8.240738156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 01:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:31:21.932743848 +0000 UTC m=+11.713850095" watchObservedRunningTime="2026-11-02 01:31:21.933398235 +0000 UTC m=+11.714504484"
Nov 02 01:31:22.049451 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:22.049387    2563 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-f6pjf" podStartSLOduration=3.926628118 podStartE2EDuration="12.049368528s" totalImagesPullingTime="8.12274041s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 01:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:31:22.048709205 +0000 UTC m=+11.829815452" watchObservedRunningTime="2026-11-02 01:31:22.049368528 +0000 UTC m=+11.830474777"
Nov 02 01:31:22.831622 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:22.830931    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52" event={"ID":"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77","Type":"ContainerStarted","Data":"f34a3c188e275cf8074098be85b3617f378d234bf6d8fee04857247ec969c8bf"}
Nov 02 01:31:22.834521 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:22.834492    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9bccs" event={"ID":"0aafee8b-ab11-4b97-afae-09b9a6d59c01","Type":"ContainerStarted","Data":"eada205b600d6d42a391a551f82022e52ae00d438b0f133bda632822e06ddef1"}
Nov 02 01:31:22.948185 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:22.948083    2563 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52" podStartSLOduration=2.962802604 podStartE2EDuration="12.948066808s" totalImagesPullingTime="9.985264204s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 01:31:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:31:22.947911153 +0000 UTC m=+12.729017400" watchObservedRunningTime="2026-11-02 01:31:22.948066808 +0000 UTC m=+12.729173057"
Nov 02 01:31:23.061812 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:23.061760    2563 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9bccs" podStartSLOduration=4.458833909 podStartE2EDuration="13.061743396s" totalImagesPullingTime="8.602909487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 01:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:31:23.060832791 +0000 UTC m=+12.841939036" watchObservedRunningTime="2026-11-02 01:31:23.061743396 +0000 UTC m=+12.842849642"
Nov 02 01:31:23.707607 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:23.707581    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:23.707749 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:23.707629    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:23.707749 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:23.707640    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:23.707749 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:23.707723    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nth82" podUID="ed8d1d91-e435-42f1-a875-f87fd18e0b87"
Nov 02 01:31:23.707851 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:23.707769    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p75zz" podUID="1092321a-6edf-43d5-bb31-5efb0b80990e"
Nov 02 01:31:23.707851 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:23.707780    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cqspb" podUID="696e3b37-d3bb-48e4-9836-afb85f9e34fd"
Nov 02 01:31:23.838542 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:23.838508    2563 generic.go:356] "Generic (PLEG): container finished" podID="f40cfbd7-701b-4fb2-8756-86ae988cb1b8" containerID="a5613059dc0cb54ad5210892f583c26b556f3ee5b1c8e6137393a1ba9fca7ffe" exitCode=0
Nov 02 01:31:23.839094 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:23.838591    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9k54v" event={"ID":"f40cfbd7-701b-4fb2-8756-86ae988cb1b8","Type":"ContainerDied","Data":"a5613059dc0cb54ad5210892f583c26b556f3ee5b1c8e6137393a1ba9fca7ffe"}
Nov 02 01:31:23.840939 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:23.840904    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-47cxd" event={"ID":"e33b967e-3f0d-49c0-bae2-c95ce8341e1f","Type":"ContainerStarted","Data":"ce73098896c19f3b128af8c4e131c5a553db97cca148ca12fc3779336c47942b"}
Nov 02 01:31:24.077419 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:24.077312    2563 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-47cxd" podStartSLOduration=2.203187046 podStartE2EDuration="13.077293405s" totalImagesPullingTime="10.874106359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 01:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:31:24.077066294 +0000 UTC m=+13.858172563" watchObservedRunningTime="2026-11-02 01:31:24.077293405 +0000 UTC m=+13.858399654"
Nov 02 01:31:25.707458 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:25.707428    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:25.707458 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:25.707441    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:25.708245 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:25.707442    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:25.708245 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:25.707528    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nth82" podUID="ed8d1d91-e435-42f1-a875-f87fd18e0b87"
Nov 02 01:31:25.708245 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:25.707609    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p75zz" podUID="1092321a-6edf-43d5-bb31-5efb0b80990e"
Nov 02 01:31:25.708245 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:25.707691    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cqspb" podUID="696e3b37-d3bb-48e4-9836-afb85f9e34fd"
Nov 02 01:31:25.847608 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:25.847571    2563 generic.go:356] "Generic (PLEG): container finished" podID="f40cfbd7-701b-4fb2-8756-86ae988cb1b8" containerID="8d114b8f6b4aa0deab45a07a2747f02249a004ed97e6977a880d3fc40cef44da" exitCode=0
Nov 02 01:31:25.847608 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:25.847611    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9k54v" event={"ID":"f40cfbd7-701b-4fb2-8756-86ae988cb1b8","Type":"ContainerDied","Data":"8d114b8f6b4aa0deab45a07a2747f02249a004ed97e6977a880d3fc40cef44da"}
Nov 02 01:31:27.627397 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:27.627365    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:27.627752 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:27.627406    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:27.627752 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:27.627475    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 01:31:27.627752 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:27.627500    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 01:31:27.627752 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:27.627531    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret podName:696e3b37-d3bb-48e4-9836-afb85f9e34fd nodeName:}" failed. No retries permitted until 2026-11-02 01:31:43.627516435 +0000 UTC m=+33.408622659 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret") pod "global-pull-secret-syncer-cqspb" (UID: "696e3b37-d3bb-48e4-9836-afb85f9e34fd") : object "kube-system"/"original-pull-secret" not registered
Nov 02 01:31:27.627752 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:27.627545    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs podName:1092321a-6edf-43d5-bb31-5efb0b80990e nodeName:}" failed. No retries permitted until 2026-11-02 01:31:43.627538857 +0000 UTC m=+33.408645080 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs") pod "network-metrics-daemon-p75zz" (UID: "1092321a-6edf-43d5-bb31-5efb0b80990e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 01:31:27.707297 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:27.707263    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:27.707297 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:27.707309    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:27.707525 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:27.707263    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:27.707525 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:27.707393    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cqspb" podUID="696e3b37-d3bb-48e4-9836-afb85f9e34fd"
Nov 02 01:31:27.707525 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:27.707467    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nth82" podUID="ed8d1d91-e435-42f1-a875-f87fd18e0b87"
Nov 02 01:31:27.707674 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:27.707541    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p75zz" podUID="1092321a-6edf-43d5-bb31-5efb0b80990e"
Nov 02 01:31:27.728522 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:27.728493    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rws7g\" (UniqueName: \"kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g\") pod \"network-check-target-nth82\" (UID: \"ed8d1d91-e435-42f1-a875-f87fd18e0b87\") " pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:27.728667 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:27.728646    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 01:31:27.728667 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:27.728665    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 01:31:27.728781 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:27.728675    2563 projected.go:196] Error preparing data for projected volume kube-api-access-rws7g for pod openshift-network-diagnostics/network-check-target-nth82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 01:31:27.728781 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:27.728722    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g podName:ed8d1d91-e435-42f1-a875-f87fd18e0b87 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:43.728703664 +0000 UTC m=+33.509809891 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rws7g" (UniqueName: "kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g") pod "network-check-target-nth82" (UID: "ed8d1d91-e435-42f1-a875-f87fd18e0b87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 01:31:28.855907 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:28.855615    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 01:31:28.856380 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:28.856206    2563 generic.go:356] "Generic (PLEG): container finished" podID="62f7a5eb-5a3c-4bd5-a0cd-486539e66c82" containerID="b93c320b687bf7b18eaabfa2e208861fbd03e06b6e09fd09bec13f5710983a2a" exitCode=1
Nov 02 01:31:28.856380 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:28.856282    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s" event={"ID":"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82","Type":"ContainerStarted","Data":"a94ad8b760992b7714b6ebdab6c6b42ab7bb8b4546c77682acf7594c3848f790"}
Nov 02 01:31:28.856380 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:28.856368    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s" event={"ID":"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82","Type":"ContainerStarted","Data":"51d5d7cae2afb7cbcade3e2525629f2f4ca1e1e05d836d5a3f5375f93bf5fa72"}
Nov 02 01:31:28.856499 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:28.856417    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s" event={"ID":"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82","Type":"ContainerStarted","Data":"e7cd9dba35321c56148c7d995ca2c58da1936694d93123e8c42899299fa9d9c5"}
Nov 02 01:31:28.856499 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:28.856465    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s" event={"ID":"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82","Type":"ContainerStarted","Data":"7c81bf876e18c3d023937e9cdf9d0bb77e5fee7d1600f5fa08dd6e786e834954"}
Nov 02 01:31:28.856499 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:28.856495    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s" event={"ID":"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82","Type":"ContainerDied","Data":"b93c320b687bf7b18eaabfa2e208861fbd03e06b6e09fd09bec13f5710983a2a"}
Nov 02 01:31:28.856616 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:28.856539    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s" event={"ID":"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82","Type":"ContainerStarted","Data":"bfcc03b2188e28488da7ee8dea5ddef972742ae284d2d7d2c615cc07e7329c91"}
Nov 02 01:31:28.858723 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:28.858705    2563 generic.go:356] "Generic (PLEG): container finished" podID="f40cfbd7-701b-4fb2-8756-86ae988cb1b8" containerID="b4380ad579458435025e7b90e9fee8d04e8d5eb4484eb61c9c2e9e8472615289" exitCode=0
Nov 02 01:31:28.858834 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:28.858732    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9k54v" event={"ID":"f40cfbd7-701b-4fb2-8756-86ae988cb1b8","Type":"ContainerDied","Data":"b4380ad579458435025e7b90e9fee8d04e8d5eb4484eb61c9c2e9e8472615289"}
Nov 02 01:31:29.706791 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:29.706755    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:29.706791 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:29.706755    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:29.706791 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:29.706784    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:29.707041 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:29.706875    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nth82" podUID="ed8d1d91-e435-42f1-a875-f87fd18e0b87"
Nov 02 01:31:29.707041 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:29.706916    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cqspb" podUID="696e3b37-d3bb-48e4-9836-afb85f9e34fd"
Nov 02 01:31:29.707041 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:29.707010    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p75zz" podUID="1092321a-6edf-43d5-bb31-5efb0b80990e"
Nov 02 01:31:29.863930 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:29.863899    2563 generic.go:356] "Generic (PLEG): container finished" podID="f40cfbd7-701b-4fb2-8756-86ae988cb1b8" containerID="31d9a0c15850681445731b663e6e32b0e5ee0c7610c88c5b0a1854fc2a7fdf0a" exitCode=0
Nov 02 01:31:29.864430 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:29.863943    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9k54v" event={"ID":"f40cfbd7-701b-4fb2-8756-86ae988cb1b8","Type":"ContainerDied","Data":"31d9a0c15850681445731b663e6e32b0e5ee0c7610c88c5b0a1854fc2a7fdf0a"}
Nov 02 01:31:30.702333 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:30.702304    2563 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hbcss"
Nov 02 01:31:30.702512 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:30.702432    2563 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 01:31:30.702902 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:30.702885    2563 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hbcss"
Nov 02 01:31:30.868951 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:30.868918    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 01:31:30.869724 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:30.869364    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s" event={"ID":"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82","Type":"ContainerStarted","Data":"29d2ab10cf915d3188bdc0dcec32e2e0d7a2f6176a68aa7370919d08ea429081"}
Nov 02 01:31:30.872531 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:30.872503    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9k54v" event={"ID":"f40cfbd7-701b-4fb2-8756-86ae988cb1b8","Type":"ContainerStarted","Data":"a601dc596369c2406164b112715057fc5fc26772dc87a077654b80298c1b1500"}
Nov 02 01:31:30.992728 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:30.992638    2563 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9k54v" podStartSLOduration=7.481127851 podStartE2EDuration="19.992625618s" totalImagesPullingTime="12.511497767s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 01:31:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:31:30.992494232 +0000 UTC m=+20.773600478" watchObservedRunningTime="2026-11-02 01:31:30.992625618 +0000 UTC m=+20.773731864"
Nov 02 01:31:31.707253 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:31.707227    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:31.707385 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:31.707303    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:31.707439 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:31.707385    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:31.707439 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:31.707385    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nth82" podUID="ed8d1d91-e435-42f1-a875-f87fd18e0b87"
Nov 02 01:31:31.707519 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:31.707496    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p75zz" podUID="1092321a-6edf-43d5-bb31-5efb0b80990e"
Nov 02 01:31:31.707557 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:31.707540    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cqspb" podUID="696e3b37-d3bb-48e4-9836-afb85f9e34fd"
Nov 02 01:31:32.879444 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:32.879419    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 01:31:32.880255 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:32.879736    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s" event={"ID":"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82","Type":"ContainerStarted","Data":"ad17e0f108a7bac0ab74a6d9a7aec918ea7d0c0e6fa74d4bc9e2be0194606373"}
Nov 02 01:31:32.880255 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:32.880042    2563 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:32.880255 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:32.880208    2563 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:32.880255 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:32.880232    2563 scope.go:122] "RemoveContainer" containerID="b93c320b687bf7b18eaabfa2e208861fbd03e06b6e09fd09bec13f5710983a2a"
Nov 02 01:31:32.880255 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:32.880257    2563 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:32.895925 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:32.895767    2563 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:32.896208 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:32.896194    2563 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:33.707581 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:33.707551    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:33.707774 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:33.707625    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:33.707774 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:33.707709    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:33.707774 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:33.707720    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cqspb" podUID="696e3b37-d3bb-48e4-9836-afb85f9e34fd"
Nov 02 01:31:33.707882 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:33.707800    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p75zz" podUID="1092321a-6edf-43d5-bb31-5efb0b80990e"
Nov 02 01:31:33.707882 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:33.707849    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nth82" podUID="ed8d1d91-e435-42f1-a875-f87fd18e0b87"
Nov 02 01:31:33.884947 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:33.884922    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 01:31:33.885453 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:33.885303    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s" event={"ID":"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82","Type":"ContainerStarted","Data":"cba9d7d9cd17ebf768784c9b09e364b06c2c54937318c2e0d5493701b7404099"}
Nov 02 01:31:34.013698 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:34.013613    2563 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s" podStartSLOduration=8.481553321 podStartE2EDuration="24.013599642s" totalImagesPullingTime="15.532046321s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 01:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:31:34.012781959 +0000 UTC m=+23.793888205" watchObservedRunningTime="2026-11-02 01:31:34.013599642 +0000 UTC m=+23.794705916"
Nov 02 01:31:35.226670 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:35.226634    2563 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cqspb"]
Nov 02 01:31:35.227253 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:35.226735    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:35.227253 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:35.226838    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cqspb" podUID="696e3b37-d3bb-48e4-9836-afb85f9e34fd"
Nov 02 01:31:35.229713 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:35.229684    2563 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nth82"]
Nov 02 01:31:35.229837 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:35.229768    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:35.229902 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:35.229857    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nth82" podUID="ed8d1d91-e435-42f1-a875-f87fd18e0b87"
Nov 02 01:31:35.230359 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:35.230331    2563 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p75zz"]
Nov 02 01:31:35.230435 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:35.230421    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:35.230547 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:35.230519    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p75zz" podUID="1092321a-6edf-43d5-bb31-5efb0b80990e"
Nov 02 01:31:36.707377 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:36.707344    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:36.707767 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:36.707442    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nth82" podUID="ed8d1d91-e435-42f1-a875-f87fd18e0b87"
Nov 02 01:31:36.707767 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:36.707350    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:36.707767 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:36.707483    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:36.707767 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:36.707553    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p75zz" podUID="1092321a-6edf-43d5-bb31-5efb0b80990e"
Nov 02 01:31:36.707767 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:36.707621    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cqspb" podUID="696e3b37-d3bb-48e4-9836-afb85f9e34fd"
Nov 02 01:31:38.707434 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:38.707404    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:38.707888 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:38.707442    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:38.707888 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:38.707524    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:38.707888 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:38.707541    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nth82" podUID="ed8d1d91-e435-42f1-a875-f87fd18e0b87"
Nov 02 01:31:38.707888 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:38.707584    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cqspb" podUID="696e3b37-d3bb-48e4-9836-afb85f9e34fd"
Nov 02 01:31:38.707888 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:38.707686    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p75zz" podUID="1092321a-6edf-43d5-bb31-5efb0b80990e"
Nov 02 01:31:40.031872 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.031797    2563 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeReady"
Nov 02 01:31:40.032350 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.031930    2563 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 02 01:31:40.144836 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.144675    2563 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lz5s9"]
Nov 02 01:31:40.146340 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.146324    2563 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dcc98"]
Nov 02 01:31:40.146463 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.146445    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lz5s9"
Nov 02 01:31:40.147902 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.147882    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:40.185785 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.185699    2563 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lz5s9"]
Nov 02 01:31:40.198317 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.198292    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 02 01:31:40.198419 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.198381    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-xjcn9\""
Nov 02 01:31:40.198419 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.198401    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 02 01:31:40.198525 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.198499    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 02 01:31:40.206353 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.206325    2563 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dcc98"]
Nov 02 01:31:40.218691 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.218664    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d49e5be4-4550-4c23-b33c-e01c22d822fe-cert\") pod \"ingress-canary-lz5s9\" (UID: \"d49e5be4-4550-4c23-b33c-e01c22d822fe\") " pod="openshift-ingress-canary/ingress-canary-lz5s9"
Nov 02 01:31:40.218781 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.218694    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2x8f\" (UniqueName: \"kubernetes.io/projected/d49e5be4-4550-4c23-b33c-e01c22d822fe-kube-api-access-h2x8f\") pod \"ingress-canary-lz5s9\" (UID: \"d49e5be4-4550-4c23-b33c-e01c22d822fe\") " pod="openshift-ingress-canary/ingress-canary-lz5s9"
Nov 02 01:31:40.218781 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.218711    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhht9\" (UniqueName: \"kubernetes.io/projected/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-kube-api-access-dhht9\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:40.218862 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.218843    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-tmp-dir\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:40.218895 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.218877    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:40.218926 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.218905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-config-volume\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:40.298676 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.298597    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 02 01:31:40.298676 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.298653    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 02 01:31:40.298874 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.298770    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6csd8\""
Nov 02 01:31:40.319196 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.319173    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-tmp-dir\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:40.319279 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.319203    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:40.319279 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.319221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-config-volume\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:40.319377 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.319286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d49e5be4-4550-4c23-b33c-e01c22d822fe-cert\") pod \"ingress-canary-lz5s9\" (UID: \"d49e5be4-4550-4c23-b33c-e01c22d822fe\") " pod="openshift-ingress-canary/ingress-canary-lz5s9"
Nov 02 01:31:40.319377 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.319313    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2x8f\" (UniqueName: \"kubernetes.io/projected/d49e5be4-4550-4c23-b33c-e01c22d822fe-kube-api-access-h2x8f\") pod \"ingress-canary-lz5s9\" (UID: \"d49e5be4-4550-4c23-b33c-e01c22d822fe\") " pod="openshift-ingress-canary/ingress-canary-lz5s9"
Nov 02 01:31:40.319377 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.319336    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhht9\" (UniqueName: \"kubernetes.io/projected/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-kube-api-access-dhht9\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:40.319377 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:40.319353    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 01:31:40.319558 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:40.319422    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls podName:8c429f46-34a2-48a4-8fe5-8204fc5ff5cb nodeName:}" failed. No retries permitted until 2026-11-02 01:31:40.819401261 +0000 UTC m=+30.600507485 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls") pod "dns-default-dcc98" (UID: "8c429f46-34a2-48a4-8fe5-8204fc5ff5cb") : secret "dns-default-metrics-tls" not found
Nov 02 01:31:40.319885 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.319863    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-tmp-dir\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:40.320046 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.320030    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-config-volume\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:40.323341 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.323324    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d49e5be4-4550-4c23-b33c-e01c22d822fe-cert\") pod \"ingress-canary-lz5s9\" (UID: \"d49e5be4-4550-4c23-b33c-e01c22d822fe\") " pod="openshift-ingress-canary/ingress-canary-lz5s9"
Nov 02 01:31:40.381040 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.381001    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhht9\" (UniqueName: \"kubernetes.io/projected/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-kube-api-access-dhht9\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:40.381241 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.381222    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2x8f\" (UniqueName: \"kubernetes.io/projected/d49e5be4-4550-4c23-b33c-e01c22d822fe-kube-api-access-h2x8f\") pod \"ingress-canary-lz5s9\" (UID: \"d49e5be4-4550-4c23-b33c-e01c22d822fe\") " pod="openshift-ingress-canary/ingress-canary-lz5s9"
Nov 02 01:31:40.459214 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.459188    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lz5s9"
Nov 02 01:31:40.708906 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.708878    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:40.709069 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.708978    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:40.709069 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.709020    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:40.761105 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.761075    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 02 01:31:40.761294 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.761274    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 02 01:31:40.761388 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.761274    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 02 01:31:40.761490 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.761477    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-m64v5\""
Nov 02 01:31:40.761595 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.761581    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fxpjk\""
Nov 02 01:31:40.761595 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.761589    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-42sf2\""
Nov 02 01:31:40.761664 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.761591    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 02 01:31:40.822721 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.822691    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:40.822865 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:40.822851    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 01:31:40.822933 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:40.822920    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls podName:8c429f46-34a2-48a4-8fe5-8204fc5ff5cb nodeName:}" failed. No retries permitted until 2026-11-02 01:31:41.822899917 +0000 UTC m=+31.604006149 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls") pod "dns-default-dcc98" (UID: "8c429f46-34a2-48a4-8fe5-8204fc5ff5cb") : secret "dns-default-metrics-tls" not found
Nov 02 01:31:40.837040 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.837009    2563 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lz5s9"]
Nov 02 01:31:40.840881 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:40.840860    2563 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd49e5be4_4550_4c23_b33c_e01c22d822fe.slice/crio-03ad0338dcfa8ea5d107b715ad2f4f457da8d0e36eefaed686b08131fe7fd2e2 WatchSource:0}: Error finding container 03ad0338dcfa8ea5d107b715ad2f4f457da8d0e36eefaed686b08131fe7fd2e2: Status 404 returned error can't find the container with id 03ad0338dcfa8ea5d107b715ad2f4f457da8d0e36eefaed686b08131fe7fd2e2
Nov 02 01:31:40.903979 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:40.903947    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lz5s9" event={"ID":"d49e5be4-4550-4c23-b33c-e01c22d822fe","Type":"ContainerStarted","Data":"03ad0338dcfa8ea5d107b715ad2f4f457da8d0e36eefaed686b08131fe7fd2e2"}
Nov 02 01:31:41.829058 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:41.828960    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:41.829058 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:41.829089    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 01:31:41.829980 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:41.829159    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls podName:8c429f46-34a2-48a4-8fe5-8204fc5ff5cb nodeName:}" failed. No retries permitted until 2026-11-02 01:31:43.829146015 +0000 UTC m=+33.610252239 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls") pod "dns-default-dcc98" (UID: "8c429f46-34a2-48a4-8fe5-8204fc5ff5cb") : secret "dns-default-metrics-tls" not found
Nov 02 01:31:43.644767 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:43.644679    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:43.644767 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:43.644751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:43.645329 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:43.644878    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 01:31:43.645329 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:43.644951    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs podName:1092321a-6edf-43d5-bb31-5efb0b80990e nodeName:}" failed. No retries permitted until 2026-11-02 01:32:15.644932195 +0000 UTC m=+65.426038434 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs") pod "network-metrics-daemon-p75zz" (UID: "1092321a-6edf-43d5-bb31-5efb0b80990e") : secret "metrics-daemon-secret" not found
Nov 02 01:31:43.647332 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:43.647303    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:43.720323 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:43.720287    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:43.745929 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:43.745897    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rws7g\" (UniqueName: \"kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g\") pod \"network-check-target-nth82\" (UID: \"ed8d1d91-e435-42f1-a875-f87fd18e0b87\") " pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:43.748211 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:43.748188    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rws7g\" (UniqueName: \"kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g\") pod \"network-check-target-nth82\" (UID: \"ed8d1d91-e435-42f1-a875-f87fd18e0b87\") " pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:43.847183 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:43.847157    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:43.847368 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:43.847344    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 01:31:43.847440 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:43.847430    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls podName:8c429f46-34a2-48a4-8fe5-8204fc5ff5cb nodeName:}" failed. No retries permitted until 2026-11-02 01:31:47.847410167 +0000 UTC m=+37.628516406 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls") pod "dns-default-dcc98" (UID: "8c429f46-34a2-48a4-8fe5-8204fc5ff5cb") : secret "dns-default-metrics-tls" not found
Nov 02 01:31:43.983446 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:43.983414    2563 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cnkn6"]
Nov 02 01:31:43.997892 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:43.997868    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:44.026302 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.026277    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:44.057582 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.057550    2563 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cnkn6"]
Nov 02 01:31:44.125056 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.125023    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qn9km\""
Nov 02 01:31:44.125056 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.125049    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 02 01:31:44.125265 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.125052    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 02 01:31:44.125265 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.125102    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 02 01:31:44.125265 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.125105    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 02 01:31:44.149381 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.149354    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:44.149464 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.149427    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:44.149566 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.149541    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6hbj\" (UniqueName: \"kubernetes.io/projected/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-api-access-r6hbj\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:44.149654 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.149634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-data-volume\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:44.149693 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.149671    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-crio-socket\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:44.217993 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.217961    2563 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-hmn9t-cwkdz"]
Nov 02 01:31:44.227847 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.227758    2563 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cqspb"]
Nov 02 01:31:44.227897 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.227868    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:44.230572 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:44.230551    2563 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod696e3b37_d3bb_48e4_9836_afb85f9e34fd.slice/crio-3dbea345989661e191b6abab611e98c5149729949b2d028fa120eecdfee3ab20 WatchSource:0}: Error finding container 3dbea345989661e191b6abab611e98c5149729949b2d028fa120eecdfee3ab20: Status 404 returned error can't find the container with id 3dbea345989661e191b6abab611e98c5149729949b2d028fa120eecdfee3ab20
Nov 02 01:31:44.249983 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.249955    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:44.250130 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.250004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:44.250130 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.250031    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6hbj\" (UniqueName: \"kubernetes.io/projected/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-api-access-r6hbj\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:44.250130 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.250058    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-data-volume\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:44.250130 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.250074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-crio-socket\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:44.250345 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:44.250128    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 01:31:44.250345 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:44.250185    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-rbac-proxy-cm podName:3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:44.750170851 +0000 UTC m=+34.531277090 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cnkn6" (UID: "3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4") : configmap "kube-rbac-proxy" not found
Nov 02 01:31:44.250345 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:44.250228    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 01:31:44.250345 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:44.250285    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls podName:3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:44.750266236 +0000 UTC m=+34.531372476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cnkn6" (UID: "3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4") : secret "insights-runtime-extractor-tls" not found
Nov 02 01:31:44.250345 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.250327    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-crio-socket\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:44.250567 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.250429    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-data-volume\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:44.278071 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.278039    2563 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-hmn9t-cwkdz"]
Nov 02 01:31:44.280358 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.280307    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 02 01:31:44.280358 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.280344    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-8qw44\""
Nov 02 01:31:44.330834 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.330797    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 02 01:31:44.333064 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.333038    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6hbj\" (UniqueName: \"kubernetes.io/projected/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-api-access-r6hbj\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:44.350896 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.350869    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fd401370-c917-4608-b6df-da7367f1ad21-archives-path\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:44.351023 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.350926    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6pv9\" (UniqueName: \"kubernetes.io/projected/fd401370-c917-4608-b6df-da7367f1ad21-kube-api-access-c6pv9\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:44.351075 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.351054    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-service-ca-bundle\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:44.351142 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.351128    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-trusted-ca-bundle\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:44.452289 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.452259    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-service-ca-bundle\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:44.452527 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.452317    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-trusted-ca-bundle\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:44.452527 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.452346    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fd401370-c917-4608-b6df-da7367f1ad21-archives-path\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:44.452527 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.452378    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6pv9\" (UniqueName: \"kubernetes.io/projected/fd401370-c917-4608-b6df-da7367f1ad21-kube-api-access-c6pv9\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:44.453003 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.452974    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fd401370-c917-4608-b6df-da7367f1ad21-archives-path\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:44.453294 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.453271    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-service-ca-bundle\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:44.454898 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.454872    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-trusted-ca-bundle\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:44.495304 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.495247    2563 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nth82"]
Nov 02 01:31:44.498476 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:44.498452    2563 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded8d1d91_e435_42f1_a875_f87fd18e0b87.slice/crio-3eae890584d18924a224d102bde27d25a8cc2511011dcad4dcc9e46648a547af WatchSource:0}: Error finding container 3eae890584d18924a224d102bde27d25a8cc2511011dcad4dcc9e46648a547af: Status 404 returned error can't find the container with id 3eae890584d18924a224d102bde27d25a8cc2511011dcad4dcc9e46648a547af
Nov 02 01:31:44.513240 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.513218    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6pv9\" (UniqueName: \"kubernetes.io/projected/fd401370-c917-4608-b6df-da7367f1ad21-kube-api-access-c6pv9\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:44.544055 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.544034    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:44.754485 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.754427    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:44.754485 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.754475    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:44.754856 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:44.754541    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 01:31:44.754856 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:44.754603    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 01:31:44.754856 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:44.754604    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-rbac-proxy-cm podName:3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:45.754588481 +0000 UTC m=+35.535694722 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cnkn6" (UID: "3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4") : configmap "kube-rbac-proxy" not found
Nov 02 01:31:44.754856 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:44.754652    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls podName:3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:45.754640559 +0000 UTC m=+35.535746784 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cnkn6" (UID: "3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4") : secret "insights-runtime-extractor-tls" not found
Nov 02 01:31:44.816486 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.816388    2563 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-2r2kf"]
Nov 02 01:31:44.848719 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.848689    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-2r2kf"
Nov 02 01:31:44.876505 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.876478    2563 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-2r2kf"]
Nov 02 01:31:44.900665 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.900642    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 02 01:31:44.900665 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.900639    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 02 01:31:44.900822 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.900804    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-846kb\""
Nov 02 01:31:44.914775 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.914748    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nth82" event={"ID":"ed8d1d91-e435-42f1-a875-f87fd18e0b87","Type":"ContainerStarted","Data":"3eae890584d18924a224d102bde27d25a8cc2511011dcad4dcc9e46648a547af"}
Nov 02 01:31:44.915862 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.915842    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cqspb" event={"ID":"696e3b37-d3bb-48e4-9836-afb85f9e34fd","Type":"ContainerStarted","Data":"3dbea345989661e191b6abab611e98c5149729949b2d028fa120eecdfee3ab20"}
Nov 02 01:31:44.917283 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.917262    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lz5s9" event={"ID":"d49e5be4-4550-4c23-b33c-e01c22d822fe","Type":"ContainerStarted","Data":"5c499d8d5cf383048097b334d78c0c9360e43188b89e19f660bb85ed524badb5"}
Nov 02 01:31:44.954421 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.954391    2563 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-hmn9t-cwkdz"]
Nov 02 01:31:44.955635 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:44.955617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p9nx\" (UniqueName: \"kubernetes.io/projected/a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24-kube-api-access-5p9nx\") pod \"migrator-66df5d496b-2r2kf\" (UID: \"a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-2r2kf"
Nov 02 01:31:44.957642 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:44.957619    2563 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd401370_c917_4608_b6df_da7367f1ad21.slice/crio-a2484711bd47b284c5c8facb3a5b7bd081e5b3315c2a3d92aa88acd422621b25 WatchSource:0}: Error finding container a2484711bd47b284c5c8facb3a5b7bd081e5b3315c2a3d92aa88acd422621b25: Status 404 returned error can't find the container with id a2484711bd47b284c5c8facb3a5b7bd081e5b3315c2a3d92aa88acd422621b25
Nov 02 01:31:45.056639 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:45.056562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p9nx\" (UniqueName: \"kubernetes.io/projected/a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24-kube-api-access-5p9nx\") pod \"migrator-66df5d496b-2r2kf\" (UID: \"a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-2r2kf"
Nov 02 01:31:45.077107 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:45.077044    2563 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lz5s9" podStartSLOduration=1.781931037 podStartE2EDuration="5.077031662s" totalImagesPullingTime="3.295100625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:31:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:31:45.076677478 +0000 UTC m=+34.857783738" watchObservedRunningTime="2026-11-02 01:31:45.077031662 +0000 UTC m=+34.858137907"
Nov 02 01:31:45.117146 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:45.117107    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p9nx\" (UniqueName: \"kubernetes.io/projected/a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24-kube-api-access-5p9nx\") pod \"migrator-66df5d496b-2r2kf\" (UID: \"a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-2r2kf"
Nov 02 01:31:45.172854 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:45.172820    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-2r2kf"
Nov 02 01:31:45.558901 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:45.558858    2563 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-2r2kf"]
Nov 02 01:31:45.563015 ip-10-0-138-75 kubenswrapper[2563]: W1102 01:31:45.562982    2563 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda00f9f5b_4aa6_4493_afb5_dd0cd6a5ac24.slice/crio-c34acf2ad1b5364ec4d1ef45f29263532d6d5f18f40f684d6901623fd5611bac WatchSource:0}: Error finding container c34acf2ad1b5364ec4d1ef45f29263532d6d5f18f40f684d6901623fd5611bac: Status 404 returned error can't find the container with id c34acf2ad1b5364ec4d1ef45f29263532d6d5f18f40f684d6901623fd5611bac
Nov 02 01:31:45.762369 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:45.762333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:45.762820 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:45.762437    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:45.762820 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:45.762516    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 01:31:45.762820 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:45.762565    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-rbac-proxy-cm podName:3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:47.762552494 +0000 UTC m=+37.543658731 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cnkn6" (UID: "3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4") : configmap "kube-rbac-proxy" not found
Nov 02 01:31:45.762996 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:45.762925    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 01:31:45.762996 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:45.762974    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls podName:3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:47.762960473 +0000 UTC m=+37.544066714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cnkn6" (UID: "3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4") : secret "insights-runtime-extractor-tls" not found
Nov 02 01:31:45.925899 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:45.925821    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-2r2kf" event={"ID":"a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24","Type":"ContainerStarted","Data":"c34acf2ad1b5364ec4d1ef45f29263532d6d5f18f40f684d6901623fd5611bac"}
Nov 02 01:31:45.927814 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:45.927779    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-hmn9t-cwkdz" event={"ID":"fd401370-c917-4608-b6df-da7367f1ad21","Type":"ContainerStarted","Data":"a2484711bd47b284c5c8facb3a5b7bd081e5b3315c2a3d92aa88acd422621b25"}
Nov 02 01:31:46.552292 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.552258    2563 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-rrqpk"]
Nov 02 01:31:46.570152 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.570111    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk"
Nov 02 01:31:46.612329 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.612300    2563 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-rrqpk"]
Nov 02 01:31:46.622383 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.622356    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-jrhl2\""
Nov 02 01:31:46.622535 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.622488    2563 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 02 01:31:46.623091 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.623046    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 02 01:31:46.623231 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.623146    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 02 01:31:46.623296 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.623243    2563 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 02 01:31:46.671805 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.671767    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d6xw\" (UniqueName: \"kubernetes.io/projected/340a686c-e0f6-4b71-a432-66659bae4e2d-kube-api-access-5d6xw\") pod \"service-ca-5fffc97bdf-rrqpk\" (UID: \"340a686c-e0f6-4b71-a432-66659bae4e2d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk"
Nov 02 01:31:46.671986 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.671956    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/340a686c-e0f6-4b71-a432-66659bae4e2d-signing-key\") pod \"service-ca-5fffc97bdf-rrqpk\" (UID: \"340a686c-e0f6-4b71-a432-66659bae4e2d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk"
Nov 02 01:31:46.672067 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.672020    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/340a686c-e0f6-4b71-a432-66659bae4e2d-signing-cabundle\") pod \"service-ca-5fffc97bdf-rrqpk\" (UID: \"340a686c-e0f6-4b71-a432-66659bae4e2d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk"
Nov 02 01:31:46.772665 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.772626    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/340a686c-e0f6-4b71-a432-66659bae4e2d-signing-key\") pod \"service-ca-5fffc97bdf-rrqpk\" (UID: \"340a686c-e0f6-4b71-a432-66659bae4e2d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk"
Nov 02 01:31:46.773157 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.772676    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/340a686c-e0f6-4b71-a432-66659bae4e2d-signing-cabundle\") pod \"service-ca-5fffc97bdf-rrqpk\" (UID: \"340a686c-e0f6-4b71-a432-66659bae4e2d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk"
Nov 02 01:31:46.773157 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.772730    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d6xw\" (UniqueName: \"kubernetes.io/projected/340a686c-e0f6-4b71-a432-66659bae4e2d-kube-api-access-5d6xw\") pod \"service-ca-5fffc97bdf-rrqpk\" (UID: \"340a686c-e0f6-4b71-a432-66659bae4e2d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk"
Nov 02 01:31:46.773920 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.773895    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/340a686c-e0f6-4b71-a432-66659bae4e2d-signing-cabundle\") pod \"service-ca-5fffc97bdf-rrqpk\" (UID: \"340a686c-e0f6-4b71-a432-66659bae4e2d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk"
Nov 02 01:31:46.776346 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.776318    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/340a686c-e0f6-4b71-a432-66659bae4e2d-signing-key\") pod \"service-ca-5fffc97bdf-rrqpk\" (UID: \"340a686c-e0f6-4b71-a432-66659bae4e2d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk"
Nov 02 01:31:46.842109 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.842039    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d6xw\" (UniqueName: \"kubernetes.io/projected/340a686c-e0f6-4b71-a432-66659bae4e2d-kube-api-access-5d6xw\") pod \"service-ca-5fffc97bdf-rrqpk\" (UID: \"340a686c-e0f6-4b71-a432-66659bae4e2d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk"
Nov 02 01:31:46.897453 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:46.897415    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk"
Nov 02 01:31:47.295131 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:47.295038    2563 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-rrqpk"]
Nov 02 01:31:47.781020 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:47.780956    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:47.781770 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:47.781039    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:47.781770 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:47.781223    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 01:31:47.781770 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:47.781289    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls podName:3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:51.781268563 +0000 UTC m=+41.562374796 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cnkn6" (UID: "3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4") : secret "insights-runtime-extractor-tls" not found
Nov 02 01:31:47.781770 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:47.781638    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 01:31:47.781770 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:47.781681    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-rbac-proxy-cm podName:3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:51.78166962 +0000 UTC m=+41.562775845 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cnkn6" (UID: "3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4") : configmap "kube-rbac-proxy" not found
Nov 02 01:31:47.882132 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:47.882086    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:47.882361 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:47.882251    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 01:31:47.882361 ip-10-0-138-75 kubenswrapper[2563]: E1102 01:31:47.882327    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls podName:8c429f46-34a2-48a4-8fe5-8204fc5ff5cb nodeName:}" failed. No retries permitted until 2026-11-02 01:31:55.882307212 +0000 UTC m=+45.663413668 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls") pod "dns-default-dcc98" (UID: "8c429f46-34a2-48a4-8fe5-8204fc5ff5cb") : secret "dns-default-metrics-tls" not found
Nov 02 01:31:47.935182 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:47.935142    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk" event={"ID":"340a686c-e0f6-4b71-a432-66659bae4e2d","Type":"ContainerStarted","Data":"18e57a24f5af00d1f37710e32856bfee51d2278ee41006201fd63db8e8717694"}
Nov 02 01:31:48.939670 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:48.939638    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-2r2kf" event={"ID":"a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24","Type":"ContainerStarted","Data":"cdf6feb2aef6884a4a535b302c46b3ad5ab6210c3bf70eaf7740a5ccafeaad81"}
Nov 02 01:31:48.942188 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:48.942143    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-hmn9t-cwkdz" event={"ID":"fd401370-c917-4608-b6df-da7367f1ad21","Type":"ContainerStarted","Data":"3ec47fccde4f4b2d63c73fd56892eaa0ec8a13d013eba2ea242acb5716c0a19c"}
Nov 02 01:31:49.054599 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:49.054539    2563 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-2r2kf" podStartSLOduration=2.4288200939999998 podStartE2EDuration="5.054522892s" totalImagesPullingTime="2.625702798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:31:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:31:49.053680005 +0000 UTC m=+38.834786272" watchObservedRunningTime="2026-11-02 01:31:49.054522892 +0000 UTC m=+38.835629139"
Nov 02 01:31:49.168943 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:49.168879    2563 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-hmn9t-cwkdz" podStartSLOduration=1.60781744 podStartE2EDuration="5.168860219s" totalImagesPullingTime="3.561042779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:31:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:31:49.168281719 +0000 UTC m=+38.949387990" watchObservedRunningTime="2026-11-02 01:31:49.168860219 +0000 UTC m=+38.949966465"
Nov 02 01:31:49.947842 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:49.947808    2563 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nth82" event={"ID":"ed8d1d91-e435-42f1-a875-f87fd18e0b87","Type":"ContainerStarted","Data":"531f90e34052b93961b8e700191945b31f6f6dc6128926f6414c7aeb27efcc22"}
Nov 02 01:31:49.948271 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:49.948162    2563 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:50.063107 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:50.063055    2563 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nth82" podStartSLOduration=35.283628609 podStartE2EDuration="40.063040378s" totalImagesPullingTime="4.779411769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:31:50.062003021 +0000 UTC m=+39.843109271" watchObservedRunningTime="2026-11-02 01:31:50.063040378 +0000 UTC m=+39.844146624"
Nov 02 01:31:50.729342 ip-10-0-138-75 kubenswrapper[2563]: I1102 01:31:50.729045    2563 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 01:31:50.729214 ip-10-0-138-75 systemd[1]: Stopping Kubernetes Kubelet...
Nov 02 01:31:50.743515 ip-10-0-138-75 systemd[1]: kubelet.service: Deactivated successfully.
Nov 02 01:31:50.743693 ip-10-0-138-75 systemd[1]: Stopped Kubernetes Kubelet.
Nov 02 01:31:50.743745 ip-10-0-138-75 systemd[1]: kubelet.service: Consumed 2.406s CPU time.
Nov 02 01:31:50.757785 ip-10-0-138-75 systemd[1]: Starting Kubernetes Kubelet...
Nov 02 01:31:50.816887 ip-10-0-138-75 kubenswrapper[4515]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 01:31:50.816887 ip-10-0-138-75 kubenswrapper[4515]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 02 01:31:50.816887 ip-10-0-138-75 kubenswrapper[4515]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 01:31:50.816887 ip-10-0-138-75 kubenswrapper[4515]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 01:31:50.818915 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818836    4515 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 01:31:50.818915 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818915    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 01:31:50.818915 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818919    4515 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818922    4515 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818925    4515 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818928    4515 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818931    4515 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818934    4515 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818937    4515 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818940    4515 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818943    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818945    4515 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818948    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818951    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818953    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818956    4515 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818959    4515 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818962    4515 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818964    4515 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818967    4515 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818969    4515 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818972    4515 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 01:31:50.819018 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818974    4515 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818977    4515 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818979    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818982    4515 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818990    4515 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818993    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818995    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.818998    4515 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819001    4515 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819004    4515 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819006    4515 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819009    4515 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819011    4515 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819014    4515 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819017    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819020    4515 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819023    4515 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819026    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819028    4515 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 01:31:50.819539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819031    4515 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819034    4515 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819036    4515 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819039    4515 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819042    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819045    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819048    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819050    4515 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819053    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819056    4515 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819059    4515 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819062    4515 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819065    4515 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819067    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819070    4515 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819073    4515 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819075    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819078    4515 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819081    4515 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 01:31:50.819997 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819084    4515 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819086    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819089    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819092    4515 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819095    4515 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819098    4515 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819100    4515 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819103    4515 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819106    4515 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819108    4515 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819110    4515 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819130    4515 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819133    4515 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819135    4515 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819138    4515 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819141    4515 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819144    4515 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819147    4515 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819151    4515 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 01:31:50.820488 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819156    4515 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819159    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819162    4515 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819165    4515 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819168    4515 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819172    4515 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819176    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819179    4515 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819183    4515 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819186    4515 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819188    4515 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819191    4515 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819194    4515 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819196    4515 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819199    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819202    4515 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819205    4515 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819208    4515 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819210    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 01:31:50.820935 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819213    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819216    4515 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819218    4515 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819221    4515 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819225    4515 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819228    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819231    4515 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.819233    4515 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820017    4515 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820024    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820028    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820031    4515 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820034    4515 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820037    4515 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820040    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820043    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820046    4515 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820048    4515 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820051    4515 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820054    4515 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 01:31:50.821434 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820057    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820059    4515 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820062    4515 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820064    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820067    4515 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820070    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820072    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820076    4515 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820081    4515 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820085    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820088    4515 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820090    4515 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820094    4515 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820098    4515 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820102    4515 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820105    4515 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820107    4515 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820110    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820128    4515 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 01:31:50.821998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820131    4515 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820133    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820136    4515 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820139    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820142    4515 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820144    4515 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820147    4515 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820150    4515 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820153    4515 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820156    4515 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820159    4515 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820161    4515 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820164    4515 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820167    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820170    4515 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820172    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820175    4515 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820178    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820181    4515 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820183    4515 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 01:31:50.822470 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820186    4515 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820191    4515 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820193    4515 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820196    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820199    4515 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820202    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820205    4515 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820207    4515 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820210    4515 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820213    4515 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820215    4515 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820218    4515 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820220    4515 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820223    4515 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820225    4515 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820228    4515 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820231    4515 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820233    4515 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820236    4515 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820239    4515 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 01:31:50.822969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820241    4515 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820244    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820247    4515 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820250    4515 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820252    4515 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820255    4515 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820257    4515 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820260    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820263    4515 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820265    4515 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820268    4515 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820270    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820273    4515 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820277    4515 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820279    4515 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820282    4515 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820285    4515 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820287    4515 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820290    4515 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820292    4515 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 01:31:50.823476 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820295    4515 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820298    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820300    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820303    4515 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820305    4515 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820308    4515 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820311    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820313    4515 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820316    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820319    4515 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820321    4515 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820324    4515 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820326    4515 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820329    4515 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820334    4515 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820806    4515 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820812    4515 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820815    4515 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820817    4515 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 01:31:50.823952 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820820    4515 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820823    4515 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820825    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820828    4515 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820832    4515 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820834    4515 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820840    4515 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820843    4515 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820846    4515 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820848    4515 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820851    4515 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820854    4515 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820856    4515 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820859    4515 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820862    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820864    4515 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820867    4515 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820870    4515 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820872    4515 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 01:31:50.824450 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820875    4515 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820879    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820882    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820884    4515 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820887    4515 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820890    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820892    4515 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820895    4515 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820898    4515 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820901    4515 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820904    4515 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820907    4515 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820909    4515 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820912    4515 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820915    4515 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820917    4515 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820920    4515 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820922    4515 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820925    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820929    4515 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 01:31:50.824913 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820931    4515 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820934    4515 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820937    4515 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820940    4515 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820942    4515 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820945    4515 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820948    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820951    4515 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820954    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820957    4515 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820959    4515 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820962    4515 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820965    4515 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820969    4515 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820972    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820975    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820977    4515 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820980    4515 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820982    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 01:31:50.825811 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820985    4515 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820988    4515 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820991    4515 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820993    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820996    4515 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.820999    4515 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821001    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821003    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821006    4515 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821009    4515 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821011    4515 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821014    4515 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821017    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821020    4515 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821023    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821025    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821028    4515 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821030    4515 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821033    4515 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 01:31:50.826414 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821036    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821040    4515 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821044    4515 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821047    4515 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821050    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821053    4515 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821056    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821059    4515 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821063    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821065    4515 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821068    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821070    4515 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821073    4515 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821077    4515 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821080    4515 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821083    4515 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821086    4515 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821089    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821091    4515 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 01:31:50.826872 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821094    4515 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821097    4515 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821100    4515 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821103    4515 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821106    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821108    4515 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821244    4515 flags.go:64] FLAG: --address="0.0.0.0"
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821254    4515 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821263    4515 flags.go:64] FLAG: --anonymous-auth="true"
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821269    4515 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821275    4515 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821280    4515 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821287    4515 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821292    4515 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821297    4515 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821301    4515 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821306    4515 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821311    4515 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821316    4515 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821321    4515 flags.go:64] FLAG: --cgroup-root=""
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821328    4515 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821333    4515 flags.go:64] FLAG: --client-ca-file=""
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821337    4515 flags.go:64] FLAG: --cloud-provider="external"
Nov 02 01:31:50.827486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821342    4515 flags.go:64] FLAG: --cluster-dns="[]"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821348    4515 flags.go:64] FLAG: --cluster-domain=""
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821352    4515 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821357    4515 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821361    4515 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821366    4515 flags.go:64] FLAG: --container-log-max-files="5"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821371    4515 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821376    4515 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821381    4515 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821386    4515 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821390    4515 flags.go:64] FLAG: --contention-profiling="false"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821395    4515 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821399    4515 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821404    4515 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821408    4515 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821417    4515 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821422    4515 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821426    4515 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821431    4515 flags.go:64] FLAG: --enable-load-reader="false"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821435    4515 flags.go:64] FLAG: --enable-server="true"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821440    4515 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821447    4515 flags.go:64] FLAG: --event-burst="100"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821452    4515 flags.go:64] FLAG: --event-qps="50"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821457    4515 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 02 01:31:50.828082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821461    4515 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821466    4515 flags.go:64] FLAG: --eviction-hard=""
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821471    4515 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821474    4515 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821477    4515 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821482    4515 flags.go:64] FLAG: --eviction-soft=""
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821485    4515 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821488    4515 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821491    4515 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821494    4515 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821497    4515 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821500    4515 flags.go:64] FLAG: --fail-swap-on="true"
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821503    4515 flags.go:64] FLAG: --feature-gates=""
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821507    4515 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821510    4515 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821513    4515 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821517    4515 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821520    4515 flags.go:64] FLAG: --healthz-port="10248"
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821523    4515 flags.go:64] FLAG: --help="false"
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821526    4515 flags.go:64] FLAG: --hostname-override="ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821530    4515 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821533    4515 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821536    4515 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 02 01:31:50.828751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821540    4515 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821544    4515 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821547    4515 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821550    4515 flags.go:64] FLAG: --image-service-endpoint=""
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821554    4515 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821559    4515 flags.go:64] FLAG: --kube-api-burst="100"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821563    4515 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821567    4515 flags.go:64] FLAG: --kube-api-qps="50"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821569    4515 flags.go:64] FLAG: --kube-reserved=""
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821572    4515 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821576    4515 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821579    4515 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821582    4515 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821584    4515 flags.go:64] FLAG: --lock-file=""
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821589    4515 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821592    4515 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821595    4515 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821600    4515 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821603    4515 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821606    4515 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821609    4515 flags.go:64] FLAG: --logging-format="text"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821612    4515 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821616    4515 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821619    4515 flags.go:64] FLAG: --manifest-url=""
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821622    4515 flags.go:64] FLAG: --manifest-url-header=""
Nov 02 01:31:50.829574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821627    4515 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821630    4515 flags.go:64] FLAG: --max-open-files="1000000"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821634    4515 flags.go:64] FLAG: --max-pods="110"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821637    4515 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821640    4515 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821643    4515 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821646    4515 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821650    4515 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821653    4515 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821656    4515 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821663    4515 flags.go:64] FLAG: --node-status-max-images="50"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821666    4515 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821669    4515 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821672    4515 flags.go:64] FLAG: --pod-cidr=""
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821675    4515 flags.go:64] FLAG: --pod-manifest-path=""
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821678    4515 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821681    4515 flags.go:64] FLAG: --pods-per-core="0"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821684    4515 flags.go:64] FLAG: --port="10250"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821687    4515 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821691    4515 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-069eb0397cf7993ea"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821694    4515 flags.go:64] FLAG: --qos-reserved=""
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821701    4515 flags.go:64] FLAG: --read-only-port="10255"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821704    4515 flags.go:64] FLAG: --register-node="true"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821707    4515 flags.go:64] FLAG: --register-with-taints=""
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821711    4515 flags.go:64] FLAG: --registry-burst="10"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821714    4515 flags.go:64] FLAG: --registry-qps="5"
Nov 02 01:31:50.830270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821717    4515 flags.go:64] FLAG: --reserved-cpus=""
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821720    4515 flags.go:64] FLAG: --reserved-memory=""
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821724    4515 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821727    4515 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821730    4515 flags.go:64] FLAG: --rotate-certificates="false"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821733    4515 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821736    4515 flags.go:64] FLAG: --runonce="false"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821739    4515 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821744    4515 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821747    4515 flags.go:64] FLAG: --seccomp-default="false"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821750    4515 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821753    4515 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821756    4515 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821760    4515 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821763    4515 flags.go:64] FLAG: --storage-driver-password="root"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821766    4515 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821769    4515 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821772    4515 flags.go:64] FLAG: --storage-driver-user="root"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821774    4515 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821778    4515 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821781    4515 flags.go:64] FLAG: --system-cgroups=""
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821784    4515 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821789    4515 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821792    4515 flags.go:64] FLAG: --tls-cert-file=""
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821795    4515 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 02 01:31:50.830909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821799    4515 flags.go:64] FLAG: --tls-min-version=""
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821802    4515 flags.go:64] FLAG: --tls-private-key-file=""
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821806    4515 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821808    4515 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821811    4515 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821814    4515 flags.go:64] FLAG: --v="2"
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821819    4515 flags.go:64] FLAG: --version="false"
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821823    4515 flags.go:64] FLAG: --vmodule=""
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821827    4515 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.821830    4515 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821966    4515 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821970    4515 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821973    4515 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821977    4515 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821980    4515 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821983    4515 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821988    4515 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821990    4515 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821993    4515 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.821996    4515 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822000    4515 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822003    4515 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 01:31:50.831539 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822007    4515 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822009    4515 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822014    4515 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822018    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822021    4515 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822024    4515 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822026    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822029    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822032    4515 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822035    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822037    4515 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822040    4515 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822044    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822047    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822049    4515 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822052    4515 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822055    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822058    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 01:31:50.832065 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822061    4515 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822065    4515 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822067    4515 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822070    4515 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822073    4515 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822075    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822078    4515 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822081    4515 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822083    4515 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822087    4515 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822089    4515 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822092    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822095    4515 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822098    4515 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822101    4515 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822103    4515 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822106    4515 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822108    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822111    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822130    4515 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 01:31:50.832544 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822133    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822136    4515 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822139    4515 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822141    4515 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822144    4515 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822147    4515 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822151    4515 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822154    4515 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822157    4515 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822160    4515 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822162    4515 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822165    4515 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822167    4515 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822170    4515 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822173    4515 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822175    4515 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822178    4515 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822181    4515 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822183    4515 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822186    4515 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 01:31:50.833019 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822188    4515 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822191    4515 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822194    4515 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822197    4515 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822200    4515 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822203    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822205    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822208    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822210    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822213    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822216    4515 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822218    4515 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822221    4515 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822223    4515 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822226    4515 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822228    4515 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822231    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822234    4515 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822237    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 01:31:50.833529 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822240    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 01:31:50.834100 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822243    4515 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 01:31:50.834100 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822245    4515 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 01:31:50.834100 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822248    4515 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 01:31:50.834100 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822251    4515 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 01:31:50.834100 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822253    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 01:31:50.834100 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822256    4515 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 01:31:50.834100 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822258    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 01:31:50.834100 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822261    4515 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 01:31:50.834100 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822265    4515 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 01:31:50.834100 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822267    4515 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 01:31:50.834100 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822270    4515 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 01:31:50.834100 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822273    4515 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 01:31:50.834100 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822275    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 01:31:50.834100 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822278    4515 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 01:31:50.834100 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822280    4515 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 01:31:50.834100 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822283    4515 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 01:31:50.834502 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.822311    4515 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 01:31:50.834502 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822345    4515 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 01:31:50.834502 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822349    4515 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 01:31:50.834502 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822352    4515 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 01:31:50.834502 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822355    4515 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 01:31:50.834502 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822358    4515 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 01:31:50.834502 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822360    4515 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 01:31:50.834502 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822363    4515 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 01:31:50.834502 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822365    4515 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 01:31:50.834502 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822368    4515 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 01:31:50.834502 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822370    4515 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 01:31:50.834502 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822373    4515 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 01:31:50.834502 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822375    4515 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 01:31:50.834502 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822378    4515 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 01:31:50.834502 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822382    4515 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 01:31:50.834502 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822385    4515 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822389    4515 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822393    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822396    4515 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822398    4515 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822401    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822403    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822406    4515 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822408    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822411    4515 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822413    4515 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822416    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822419    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822421    4515 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822424    4515 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822427    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822429    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822434    4515 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 01:31:50.834886 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822438    4515 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822441    4515 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822444    4515 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822447    4515 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822449    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822452    4515 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822454    4515 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822457    4515 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822460    4515 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822462    4515 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822465    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822467    4515 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822470    4515 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822474    4515 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822477    4515 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822480    4515 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822483    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822485    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822488    4515 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822491    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 01:31:50.835430 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822493    4515 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822496    4515 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822499    4515 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822501    4515 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822504    4515 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822506    4515 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822510    4515 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822513    4515 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822515    4515 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822518    4515 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822521    4515 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822524    4515 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822527    4515 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822530    4515 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822532    4515 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822535    4515 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822538    4515 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822540    4515 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822543    4515 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822545    4515 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 01:31:50.836012 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822548    4515 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822551    4515 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822553    4515 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822556    4515 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822558    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822562    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822565    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822567    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822569    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822572    4515 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822575    4515 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822577    4515 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822580    4515 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822582    4515 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822585    4515 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822587    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822590    4515 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822592    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822596    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 01:31:50.836545 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822599    4515 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822601    4515 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822604    4515 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822607    4515 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822609    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822612    4515 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822615    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822618    4515 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822620    4515 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822623    4515 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822626    4515 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822628    4515 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822631    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822634    4515 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.822637    4515 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.827508    4515 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.827526    4515 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.827545    4515 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.827550    4515 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 02 01:31:50.836994 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827608    4515 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827614    4515 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827617    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827620    4515 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827623    4515 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827625    4515 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827628    4515 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827631    4515 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827634    4515 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827637    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827639    4515 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827642    4515 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827646    4515 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827649    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827651    4515 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827655    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827657    4515 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827660    4515 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827663    4515 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827666    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 01:31:50.837467 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827668    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827671    4515 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827673    4515 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827676    4515 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827679    4515 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827683    4515 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827686    4515 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827690    4515 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827697    4515 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827701    4515 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827704    4515 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827706    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827709    4515 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827712    4515 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827714    4515 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827717    4515 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827720    4515 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827722    4515 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827725    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 01:31:50.837957 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827728    4515 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827730    4515 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827733    4515 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827736    4515 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827738    4515 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827741    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827744    4515 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827747    4515 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827750    4515 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827752    4515 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827755    4515 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827758    4515 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827760    4515 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827763    4515 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827766    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827768    4515 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827771    4515 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827773    4515 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827776    4515 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827779    4515 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 01:31:50.838432 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827782    4515 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827784    4515 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827787    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827790    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827792    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827795    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827798    4515 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827800    4515 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827803    4515 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827805    4515 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827808    4515 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827811    4515 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827813    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827816    4515 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827818    4515 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827821    4515 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827824    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827828    4515 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827832    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 01:31:50.838908 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827834    4515 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827838    4515 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827841    4515 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827843    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827846    4515 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827849    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827851    4515 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827854    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827857    4515 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827860    4515 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827862    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827865    4515 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827867    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827870    4515 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827873    4515 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827876    4515 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827878    4515 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827881    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827884    4515 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 01:31:50.839426 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827887    4515 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 01:31:50.839873 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827889    4515 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 01:31:50.839873 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827892    4515 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 01:31:50.839873 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827895    4515 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 01:31:50.839873 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827897    4515 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 01:31:50.839873 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827900    4515 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 01:31:50.839873 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827902    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 01:31:50.839873 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827905    4515 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 01:31:50.839873 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.827908    4515 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 01:31:50.839873 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.827935    4515 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 01:31:50.839873 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828080    4515 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 01:31:50.839873 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828085    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 01:31:50.839873 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828088    4515 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 01:31:50.839873 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828091    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 01:31:50.839873 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828094    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 01:31:50.839873 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828098    4515 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 01:31:50.839873 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828101    4515 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828103    4515 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828106    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828109    4515 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828111    4515 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828134    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828138    4515 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828141    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828143    4515 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828146    4515 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828149    4515 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828152    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828154    4515 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828157    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828160    4515 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828163    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828165    4515 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828168    4515 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828171    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 01:31:50.840311 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828173    4515 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828176    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828179    4515 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828181    4515 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828184    4515 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828187    4515 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828190    4515 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828192    4515 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828195    4515 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828200    4515 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828202    4515 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828206    4515 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828208    4515 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828212    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828215    4515 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828217    4515 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828220    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828223    4515 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828225    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828228    4515 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 01:31:50.840759 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828231    4515 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828233    4515 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828236    4515 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828239    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828241    4515 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828244    4515 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828246    4515 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828249    4515 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828251    4515 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828254    4515 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828257    4515 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828260    4515 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828262    4515 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828265    4515 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828267    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828271    4515 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828275    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828278    4515 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828281    4515 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 01:31:50.841269 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828284    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828286    4515 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828290    4515 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828293    4515 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828296    4515 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828299    4515 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828301    4515 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828305    4515 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828309    4515 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828314    4515 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828317    4515 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828320    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828323    4515 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828325    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828328    4515 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828332    4515 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828335    4515 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828338    4515 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828340    4515 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828343    4515 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 01:31:50.841742 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828346    4515 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828348    4515 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828351    4515 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828354    4515 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828356    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828359    4515 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828362    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828364    4515 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828367    4515 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828369    4515 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828372    4515 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828375    4515 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828377    4515 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828380    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828384    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828387    4515 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828389    4515 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828392    4515 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828395    4515 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828398    4515 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 01:31:50.842253 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828401    4515 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 01:31:50.842729 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828404    4515 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 01:31:50.842729 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.828428    4515 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 01:31:50.842729 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828460    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 01:31:50.842729 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828464    4515 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 01:31:50.842729 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828468    4515 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 01:31:50.842729 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828471    4515 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 01:31:50.842729 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828475    4515 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 01:31:50.842729 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828478    4515 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 01:31:50.842729 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828481    4515 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 01:31:50.842729 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828483    4515 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 01:31:50.842729 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828486    4515 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 01:31:50.842729 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828489    4515 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 01:31:50.842729 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828492    4515 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 01:31:50.842729 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828494    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 01:31:50.842729 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828497    4515 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 01:31:50.842729 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828500    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828503    4515 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828506    4515 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828508    4515 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828511    4515 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828514    4515 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828517    4515 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828519    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828522    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828524    4515 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828527    4515 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828531    4515 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828534    4515 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828536    4515 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828539    4515 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828542    4515 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828545    4515 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828548    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828551    4515 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 01:31:50.843259 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828553    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828556    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828559    4515 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828562    4515 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828565    4515 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828568    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828571    4515 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828574    4515 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828576    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828579    4515 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828582    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828585    4515 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828587    4515 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828590    4515 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828593    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828595    4515 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828598    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828600    4515 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828603    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 01:31:50.843701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828606    4515 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828608    4515 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828611    4515 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828614    4515 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828616    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828623    4515 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828626    4515 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828629    4515 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828631    4515 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828634    4515 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828637    4515 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828639    4515 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828643    4515 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828646    4515 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828649    4515 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828652    4515 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828655    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828659    4515 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828661    4515 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828664    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 01:31:50.844182 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828666    4515 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828669    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828672    4515 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828674    4515 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828677    4515 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828680    4515 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828682    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828685    4515 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828687    4515 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828690    4515 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828693    4515 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828695    4515 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828698    4515 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828701    4515 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828703    4515 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828706    4515 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828708    4515 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828712    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828716    4515 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 01:31:50.844645 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828719    4515 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828722    4515 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828725    4515 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828728    4515 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828731    4515 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828733    4515 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828736    4515 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828739    4515 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828742    4515 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828744    4515 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828748    4515 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828752    4515 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828756    4515 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828760    4515 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828762    4515 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:50.828766    4515 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.828824    4515 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.829948    4515 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.830008    4515 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 02 01:31:50.845097 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.830200    4515 server.go:1014] "Starting client certificate rotation"
Nov 02 01:31:50.845575 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.830336    4515 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 01:31:50.845575 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.830378    4515 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-01 01:26:10 +0000 UTC" deadline="2028-07-25 15:08:18.892789032 +0000 UTC"
Nov 02 01:31:50.845575 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.830407    4515 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15157h36m28.062384198s"
Nov 02 01:31:50.845575 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.830639    4515 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 01:31:50.850554 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.850531    4515 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 01:31:50.852012 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.851996    4515 log.go:25] "Validated CRI v1 runtime API"
Nov 02 01:31:50.853596 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.853580    4515 log.go:25] "Validated CRI v1 image API"
Nov 02 01:31:50.854321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.854304    4515 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 02 01:31:50.856253 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.856229    4515 fs.go:135] Filesystem UUIDs: map[4fb30ead-4668-44ea-8407-816ebc85e31c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a2df9fa6-7dd1-4f64-b662-7bf7cd9195b5:/dev/nvme0n1p3]
Nov 02 01:31:50.856422 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.856252    4515 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/03ad0338dcfa8ea5d107b715ad2f4f457da8d0e36eefaed686b08131fe7fd2e2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/03ad0338dcfa8ea5d107b715ad2f4f457da8d0e36eefaed686b08131fe7fd2e2/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/18e57a24f5af00d1f37710e32856bfee51d2278ee41006201fd63db8e8717694/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/18e57a24f5af00d1f37710e32856bfee51d2278ee41006201fd63db8e8717694/userdata/shm major:0 minor:235 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1a1a99bc627340193bbd40fe1a5b38f3cb0e46cdccd5ce51adea4f27cba68b5e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1a1a99bc627340193bbd40fe1a5b38f3cb0e46cdccd5ce51adea4f27cba68b5e/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/21120f85e79b6dddbb9c5ff481c813160558def131429c9f93bffbbfa01ee947/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/21120f85e79b6dddbb9c5ff481c813160558def131429c9f93bffbbfa01ee947/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3dbea345989661e191b6abab611e98c5149729949b2d028fa120eecdfee3ab20/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3dbea345989661e191b6abab611e98c5149729949b2d028fa120eecdfee3ab20/userdata/shm major:0 minor:208 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3eae890584d18924a224d102bde27d25a8cc2511011dcad4dcc9e46648a547af/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3eae890584d18924a224d102bde27d25a8cc2511011dcad4dcc9e46648a547af/userdata/shm major:0 minor:210 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/819ee76eaddc5e64334fc9b9cf78fd4e98a5f24bb9338793fe78562a091f87e3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/819ee76eaddc5e64334fc9b9cf78fd4e98a5f24bb9338793fe78562a091f87e3/userdata/shm major:0 minor:63 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/982ddaade8fa1cf87d9a02fbaff8ccf6d1c21e5e30027a5ded433d0ec1fc1c24/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/982ddaade8fa1cf87d9a02fbaff8ccf6d1c21e5e30027a5ded433d0ec1fc1c24/userdata/shm major:0 minor:62 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/98e4014dc2542786d0aa2a7ccace60f106061391d004e725a0ce667b45cdfbab/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/98e4014dc2542786d0aa2a7ccace60f106061391d004e725a0ce667b45cdfbab/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a2484711bd47b284c5c8facb3a5b7bd081e5b3315c2a3d92aa88acd422621b25/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a2484711bd47b284c5c8facb3a5b7bd081e5b3315c2a3d92aa88acd422621b25/userdata/shm major:0 minor:224 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b6a21ea7b12c785fcee3884c1c9e1539ba4d90fca5283b36f93bebd60ba0ca38/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b6a21ea7b12c785fcee3884c1c9e1539ba4d90fca5283b36f93bebd60ba0ca38/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c34acf2ad1b5364ec4d1ef45f29263532d6d5f18f40f684d6901623fd5611bac/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c34acf2ad1b5364ec4d1ef45f29263532d6d5f18f40f684d6901623fd5611bac/userdata/shm major:0 minor:229 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d1e73f09594b5e38baccb49b8c5d3d2936ef28ad507f41fa1d8db246627b9a16/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d1e73f09594b5e38baccb49b8c5d3d2936ef28ad507f41fa1d8db246627b9a16/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ded9e93ebf8ca912e18a1a7f8bfd8ec8d008e1fe8cca740687bc5b3ecc4af381/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ded9e93ebf8ca912e18a1a7f8bfd8ec8d008e1fe8cca740687bc5b3ecc4af381/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e896a197f81fdba8c0f0e3a2b999f02eecbcfc614349e74b00e3c79f2fec99aa/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e896a197f81fdba8c0f0e3a2b999f02eecbcfc614349e74b00e3c79f2fec99aa/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ecb4de11d1b780ab7bdc06ace820e4fae200e440969d25b6b1e2f1e1c402d672/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ecb4de11d1b780ab7bdc06ace820e4fae200e440969d25b6b1e2f1e1c402d672/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0923847d-508c-4308-93f9-543d9bb3c251/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/0923847d-508c-4308-93f9-543d9bb3c251/volumes/kubernetes.io~secret/agent-certs major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aafee8b-ab11-4b97-afae-09b9a6d59c01/volumes/kubernetes.io~projected/kube-api-access-dfc7g:{mountpoint:/var/lib/kubelet/pods/0aafee8b-ab11-4b97-afae-09b9a6d59c01/volumes/kubernetes.io~projected/kube-api-access-dfc7g major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1092321a-6edf-43d5-bb31-5efb0b80990e/volumes/kubernetes.io~projected/kube-api-access-skmsl:{mountpoint:/var/lib/kubelet/pods/1092321a-6edf-43d5-bb31-5efb0b80990e/volumes/kubernetes.io~projected/kube-api-access-skmsl major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/340a686c-e0f6-4b71-a432-66659bae4e2d/volumes/kubernetes.io~projected/kube-api-access-5d6xw:{mountpoint:/var/lib/kubelet/pods/340a686c-e0f6-4b71-a432-66659bae4e2d/volumes/kubernetes.io~projected/kube-api-access-5d6xw major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/340a686c-e0f6-4b71-a432-66659bae4e2d/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/340a686c-e0f6-4b71-a432-66659bae4e2d/volumes/kubernetes.io~secret/signing-key major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4/volumes/kubernetes.io~projected/kube-api-access-r6hbj:{mountpoint:/var/lib/kubelet/pods/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4/volumes/kubernetes.io~projected/kube-api-access-r6hbj major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/414870a7-ef47-4b1e-aac2-1b25923286e9/volumes/kubernetes.io~projected/kube-api-access-2kctw:{mountpoint:/var/lib/kubelet/pods/414870a7-ef47-4b1e-aac2-1b25923286e9/volumes/kubernetes.io~projected/kube-api-access-2kctw major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/50bfe6af-769e-40aa-8a09-560ea1a984a2/volumes/kubernetes.io~projected/kube-api-access-jrjqz:{mountpoint:/var/lib/kubelet/pods/50bfe6af-769e-40aa-8a09-560ea1a984a2/volumes/kubernetes.io~projected/kube-api-access-jrjqz major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77/volumes/kubernetes.io~projected/kube-api-access-vsdhq:{mountpoint:/var/lib/kubelet/pods/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77/volumes/kubernetes.io~projected/kube-api-access-vsdhq major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/volumes/kubernetes.io~projected/kube-api-access-pjz92:{mountpoint:/var/lib/kubelet/pods/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/volumes/kubernetes.io~projected/kube-api-access-pjz92 major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/696e3b37-d3bb-48e4-9836-afb85f9e34fd/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/696e3b37-d3bb-48e4-9836-afb85f9e34fd/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb/volumes/kubernetes.io~projected/kube-api-access-dhht9:{mountpoint:/var/lib/kubelet/pods/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb/volumes/kubernetes.io~projected/kube-api-access-dhht9 major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24/volumes/kubernetes.io~projected/kube-api-access-5p9nx:{mountpoint:/var/lib/kubelet/pods/a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24/volumes/kubernetes.io~projected/kube-api-access-5p9nx major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db/volumes/kubernetes.io~projected/kube-api-access-mdkwl:{mountpoint:/var/lib/kubelet/pods/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db/volumes/kubernetes.io~projected/kube-api-access-mdkwl major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d49e5be4-4550-4c23-b33c-e01c22d822fe/volumes/kubernetes.io~projected/kube-api-access-h2x8f:{mountpoint:/var/lib/kubelet/pods/d49e5be4-4550-4c23-b33c-e01c22d822fe/volumes/kubernetes.io~projected/kube-api-access-h2x8f major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d49e5be4-4550-4c23-b33c-e01c22d822fe/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/d49e5be4-4550-4c23-b33c-e01c22d822fe/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e33b967e-3f0d-49c0-bae2-c95ce8341e1f/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/e33b967e-3f0d-49c0-bae2-c95ce8341e1f/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e33b967e-3f0d-49c0-bae2-c95ce8341e1f/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/e33b967e-3f0d-49c0-bae2-c95ce8341e1f/volumes/kubernetes.io~empty-dir/tmp major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e33b967e-3f0d-49c0-bae2-c95ce8341e1f/volumes/kubernetes.io~projected/kube-api-access-gjsm4:{mountpoint:/var/lib/kubelet/pods/e33b967e-3f0d-49c0-bae2-c95ce8341e1f/volumes/kubernetes.io~projected/kube-api-access-gjsm4 major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed8d1d91-e435-42f1-a875-f87fd18e0b87/volumes/kubernetes.io~projected/kube-api-access-rws7g:{mountpoint:/var/lib/kubelet/pods/ed8d1d91-e435-42f1-a875-f87fd18e0b87/volumes/kubernetes.io~projected/kube-api-access-rws7g major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f40cfbd7-701b-4fb2-8756-86ae988cb1b8/volumes/kubernetes.io~projected/kube-api-access-99m65:{mountpoint:/var/lib/kubelet/pods/f40cfbd7-701b-4fb2-8756-86ae988cb1b8/volumes/kubernetes.io~projected/kube-api-access-99m65 major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fd401370-c917-4608-b6df-da7367f1ad21/volumes/kubernetes.io~projected/kube-api-access-c6pv9:{mountpoint:/var/lib/kubelet/pods/fd401370-c917-4608-b6df-da7367f1ad21/volumes/kubernetes.io~projected/kube-api-access-c6pv9 major:0 minor:223 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/103f5184e0696b8802de12ea3502ae655260d9eb0b954a767bd9dcab90fa762a/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/36c607f12bbb90fe6f4ec35a4a7389628b3ff107a5bec13dd468e95cd966a250/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/3e1104baaa9df2d7b89e7c56022ea23979f9631dd9a7dbc7886983e8748d1fd7/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/98247d35586f628619fcc03514c3343bae3a64ad11d85746973dc862a2e0b64a/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-111:{mountpoint:/var/lib/containers/storage/overlay/466689ee360b9dcf154cc94adc69dc5abdaecf8e9283d02995a747f43b034a25/merged major:0 minor:111 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/ea226adaaf3e25628e4da5468fd94e4b02bdb3e9cc1866dc5fc07df2989f3b74/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/3bec1c73f15f9feb58e82b7aa5335e4fda81dacf9bdf4aacdff4a5392efdad69/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/21af22a20c8752440e20ec32dd3e703eb7fccc1eb62eb8dd187ee12191476523/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/52c6591b0309477e98e8e410619e83513c2be9d11c628c88ee505b22a54d08de/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-131:{mountpoint:/var/lib/containers/storage/overlay/b520975e4201e321733906d71d8145937fba7f76fa00bac7001d718860dc1517/merged major:0 minor:131 fsType:overlay blockSize:0} overlay_0-143:{mountpoint:/var/lib/containers/storage/overlay/d97985782483f0b9e63a32c5f61e9f1209b4a71f429446d3207ce8539d8fce04/merged major:0 minor:143 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/6243ca0024e9ba6896408c2a340c6f2da76821f9bb2abe6e71e5a7cfe4398b7c/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/da0bda95b3f36c0753a0734a52d1b84fafeec0f8c911b9dedb761ab789861c5d/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/72abc78bea04e9ed2c942a45592ac799b2e64acff7401071b4cd4162f1f541ca/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/38f437134f28901e0d18c558d5ac429f44f196b6ca1f0080c06d264def7fc504/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/fe1a2a6980e72814e5c0b4d1624d011eacd400c1bb9b8a4deb346f77bf9909f7/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/39f8073ef5c87dcdd1973a349cf61941dbd7e56d5217fc7184566424586bc193/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/074209ee3d3b78033420c3264824aa3c083f19dc3875929142ca479c7777f03b/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-179:{mountpoint:/var/lib/containers/storage/overlay/aae68073f96b4273db4485ba409a7b45ef27fece594da9a39b753fbbb7eed594/merged major:0 minor:179 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/097b361460e0640cac95e0f10cb930c8b8b80a9d6b6bbd5cdcfc05d0221bd489/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/c732d8ade153f9d83246c738a423520c57c4543c30718b1d54ca19682b8230af/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-204:{mountpoint:/var/lib/containers/storage/overlay/6e8e27c72ad25a5948bf2fe72cb3bd4a00e0864d5b7a81e6d2a754af9a17dd43/merged major:0 minor:204 fsType:overlay blockSize:0} overlay_0-212:{mountpoint:/var/lib/containers/storage/overlay/42d4359d5ff16ce96eff9fbe2a9c286467f47618dd1f1bc4e6fa3f101fc66cd8/merged major:0 minor:212 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/3a47976f593564dd628f858b75681a5e3dc26ad00c0a8481991a1fa024d31188/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-221:{mountpoint:/var/lib/containers/storage/overlay/53988e837c584f4ff5a8cc2e6370c35754025b4ffb2e8822dc79c899893b74ce/merged major:0 minor:221 fsType:overlay blockSize:0} overlay_0-226:{mountpoint:/var/lib/containers/storage/overlay/a333f689680bc97386688633b2df413eef767ae2fe4296b00f3f566ed4c41d98/merged major:0 minor:226 fsType:overlay blockSize:0} overlay_0-231:{mountpoint:/var/lib/containers/storage/overlay/2ee88d07289aec743993a4175620fec30fbaabb973c1f61679d5062cc250d22a/merged major:0 minor:231 fsType:overlay blockSize:0} overlay_0-237:{mountpoint:/var/lib/containers/storage/overlay/f4acdda9398c2431a74fc77683548a0c01b08977d9e9035ca6f462d9ed9f5476/merged major:0 minor:237 fsType:overlay blockSize:0} overlay_0-239:{mountpoint:/var/lib/containers/storage/overlay/ba6b830f3ca75111ff0b1e21d5b2fcab1ea30fa1207f64b7784724464e64d8ac/merged major:0 minor:239 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/d032c4ae84ac40142a6932215945b750cef78bf9eeead2d1ceb3efc156700685/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/4a04c8d75654a2c1a37ca478fdbf52abd350091d88c6f3e81bbf54c93ba3c2af/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/8f847cc2149af195f6841fd1428db3fd151b3230079b248427dbb352c9a6b517/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/c0c1db0f1d74523579cb30be0eaeb2b02b2eb6519373f4a12bf532bfa0517ffc/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/2d065a054e07e5b5d45476f9d471626a1f0af964a3f47211a2f937cd428b32fa/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/9492f77380ad174dc16889db7d395ac85603c73e7f884224bddc6e80d488b9e8/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/f28dde06ffee9ec66d2b70b09036951ccf692342c938639a9188265c2e33f466/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/a510851346cd3de53c5e6de9d10e326f81dc5451135fc85e159d463b7ec18a8c/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/d4628533959a5789a44a4c61784334ffc9cfb04939ce8e5a05178474e4d24bab/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/916cd6e633db566615697d735b34c53cc31005c1a67f46354f43614488823068/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/11a27ad7de71ef88c62b43f3b93753473c70c43fd141342abaeacf7ef8de77ae/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/6963e726e01c39c79111cf64b0949617fa825469acf78c9364b693d63f5acb16/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/e1b53f6af072ead74fb88e72957156923b9022a5f7468b88bab7850e8219a863/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 02 01:31:50.865173 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.864878    4515 manager.go:217] Machine: {Timestamp:2026-11-02 01:31:50.86442268 +0000 UTC m=+0.076792479 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099402 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d2cf2c1e7f59217e112280a3e32e0 SystemUUID:ec2d2cf2-c1e7-f592-17e1-12280a3e32e0 BootID:4db775b0-7797-4307-ba92-ee34e4d47977 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3dbea345989661e191b6abab611e98c5149729949b2d028fa120eecdfee3ab20/userdata/shm DeviceMajor:0 DeviceMinor:208 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-221 DeviceMajor:0 DeviceMinor:221 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/414870a7-ef47-4b1e-aac2-1b25923286e9/volumes/kubernetes.io~projected/kube-api-access-2kctw DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4/volumes/kubernetes.io~projected/kube-api-access-r6hbj DeviceMajor:0 DeviceMinor:214 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a2484711bd47b284c5c8facb3a5b7bd081e5b3315c2a3d92aa88acd422621b25/userdata/shm DeviceMajor:0 DeviceMinor:224 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24/volumes/kubernetes.io~projected/kube-api-access-5p9nx DeviceMajor:0 DeviceMinor:228 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-237 DeviceMajor:0 DeviceMinor:237 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/50bfe6af-769e-40aa-8a09-560ea1a984a2/volumes/kubernetes.io~projected/kube-api-access-jrjqz DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/98e4014dc2542786d0aa2a7ccace60f106061391d004e725a0ce667b45cdfbab/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e33b967e-3f0d-49c0-bae2-c95ce8341e1f/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:49 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77/volumes/kubernetes.io~projected/kube-api-access-vsdhq DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb/volumes/kubernetes.io~projected/kube-api-access-dhht9 DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/03ad0338dcfa8ea5d107b715ad2f4f457da8d0e36eefaed686b08131fe7fd2e2/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:47 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f40cfbd7-701b-4fb2-8756-86ae988cb1b8/volumes/kubernetes.io~projected/kube-api-access-99m65 DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1a1a99bc627340193bbd40fe1a5b38f3cb0e46cdccd5ce51adea4f27cba68b5e/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/696e3b37-d3bb-48e4-9836-afb85f9e34fd/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:206 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/340a686c-e0f6-4b71-a432-66659bae4e2d/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:233 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/18e57a24f5af00d1f37710e32856bfee51d2278ee41006201fd63db8e8717694/userdata/shm DeviceMajor:0 DeviceMinor:235 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/340a686c-e0f6-4b71-a432-66659bae4e2d/volumes/kubernetes.io~projected/kube-api-access-5d6xw DeviceMajor:0 DeviceMinor:234 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-179 DeviceMajor:0 DeviceMinor:179 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3eae890584d18924a224d102bde27d25a8cc2511011dcad4dcc9e46648a547af/userdata/shm DeviceMajor:0 DeviceMinor:210 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/e33b967e-3f0d-49c0-bae2-c95ce8341e1f/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:48 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ded9e93ebf8ca912e18a1a7f8bfd8ec8d008e1fe8cca740687bc5b3ecc4af381/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-204 DeviceMajor:0 DeviceMinor:204 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-226 DeviceMajor:0 DeviceMinor:226 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c34acf2ad1b5364ec4d1ef45f29263532d6d5f18f40f684d6901623fd5611bac/userdata/shm DeviceMajor:0 DeviceMinor:229 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/volumes/kubernetes.io~projected/kube-api-access-pjz92 DeviceMajor:0 DeviceMinor:54 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d49e5be4-4550-4c23-b33c-e01c22d822fe/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0923847d-508c-4308-93f9-543d9bb3c251/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:50 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/21120f85e79b6dddbb9c5ff481c813160558def131429c9f93bffbbfa01ee947/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fd401370-c917-4608-b6df-da7367f1ad21/volumes/kubernetes.io~projected/kube-api-access-c6pv9 DeviceMajor:0 DeviceMinor:223 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e33b967e-3f0d-49c0-bae2-c95ce8341e1f/volumes/kubernetes.io~projected/kube-api-access-gjsm4 DeviceMajor:0 DeviceMinor:51 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ecb4de11d1b780ab7bdc06ace820e4fae200e440969d25b6b1e2f1e1c402d672/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e896a197f81fdba8c0f0e3a2b999f02eecbcfc614349e74b00e3c79f2fec99aa/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b6a21ea7b12c785fcee3884c1c9e1539ba4d90fca5283b36f93bebd60ba0ca38/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-212 DeviceMajor:0 DeviceMinor:212 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-239 DeviceMajor:0 DeviceMinor:239 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0aafee8b-ab11-4b97-afae-09b9a6d59c01/volumes/kubernetes.io~projected/kube-api-access-dfc7g DeviceMajor:0 DeviceMinor:52 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db/volumes/kubernetes.io~projected/kube-api-access-mdkwl DeviceMajor:0 DeviceMinor:53 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/982ddaade8fa1cf87d9a02fbaff8ccf6d1c21e5e30027a5ded433d0ec1fc1c24/userdata/shm DeviceMajor:0 DeviceMinor:62 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-131 DeviceMajor:0 DeviceMinor:131 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-143 DeviceMajor:0 DeviceMinor:143 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ed8d1d91-e435-42f1-a875-f87fd18e0b87/volumes/kubernetes.io~projected/kube-api-access-rws7g DeviceMajor:0 DeviceMinor:207 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-231 DeviceMajor:0 DeviceMinor:231 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1092321a-6edf-43d5-bb31-5efb0b80990e/volumes/kubernetes.io~projected/kube-api-access-skmsl DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d1e73f09594b5e38baccb49b8c5d3d2936ef28ad507f41fa1d8db246627b9a16/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-111 DeviceMajor:0 DeviceMinor:111 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d49e5be4-4550-4c23-b33c-e01c22d822fe/volumes/kubernetes.io~projected/kube-api-access-h2x8f DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/819ee76eaddc5e64334fc9b9cf78fd4e98a5f24bb9338793fe78562a091f87e3/userdata/shm DeviceMajor:0 DeviceMinor:63 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:03ad0338dcfa8ea MacAddress:46:26:6a:fc:33:a8 Speed:10000 Mtu:8901} {Name:18e57a24f5af00d MacAddress:a6:e9:70:20:85:a1 Speed:10000 Mtu:8901} {Name:3dbea345989661e MacAddress:6a:72:1b:c9:c9:41 Speed:10000 Mtu:8901} {Name:3eae890584d1892 MacAddress:aa:d7:d6:10:34:98 Speed:10000 Mtu:8901} {Name:a2484711bd47b28 MacAddress:62:33:21:b0:68:ef Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:56:ca:d2:fc:b5 Speed:0 Mtu:9001} {Name:br-int MacAddress:6a:1d:8b:2c:ba:45 Speed:0 Mtu:8901} {Name:c34acf2ad1b5364 MacAddress:42:36:eb:ed:0a:b4 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:56:ca:d2:fc:b5 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:d2:91:a0:19:7a:6d Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:9e:a1:a2:fd:bf:00 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 02 01:31:50.865173 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.865173    4515 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 02 01:31:50.865420 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.865222    4515 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 02 01:31:50.865564 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.865533    4515 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 02 01:31:50.865694 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.865564    4515 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-75.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 02 01:31:50.865739 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.865697    4515 topology_manager.go:143] "Creating topology manager with none policy"
Nov 02 01:31:50.865739 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.865705    4515 container_manager_linux.go:311] "Creating device plugin manager"
Nov 02 01:31:50.865739 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.865715    4515 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 01:31:50.865739 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.865735    4515 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 01:31:50.865841 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.865800    4515 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 02 01:31:50.865969 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.865960    4515 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 02 01:31:50.866093 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.866084    4515 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 02 01:31:50.866153 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.866145    4515 kubelet.go:493] "Attempting to sync node with API server"
Nov 02 01:31:50.866183 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.866158    4515 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 02 01:31:50.866183 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.866169    4515 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 02 01:31:50.866183 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.866177    4515 kubelet.go:405] "Adding apiserver pod source"
Nov 02 01:31:50.866282 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.866185    4515 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 02 01:31:50.866810 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.866798    4515 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 02 01:31:50.866853 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.866811    4515 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 01:31:50.867366 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.867344    4515 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 02 01:31:50.911574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.911550    4515 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 02 01:31:50.911574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.911566    4515 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 01:31:50.911716 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.911681    4515 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 02 01:31:50.911885 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.911874    4515 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 02 01:31:50.911916 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.911904    4515 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 02 01:31:50.912018 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.912008    4515 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 02 01:31:50.912054 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.912023    4515 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 02 01:31:50.912054 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.912030    4515 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 02 01:31:50.912054 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.912035    4515 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 02 01:31:50.912054 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.912043    4515 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 02 01:31:50.912054 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.912049    4515 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 02 01:31:50.912291 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.912059    4515 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 02 01:31:50.912291 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.912068    4515 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 02 01:31:50.912291 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.912076    4515 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 02 01:31:50.912291 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.912082    4515 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 02 01:31:50.912291 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.912096    4515 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 02 01:31:50.912291 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.912106    4515 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 02 01:31:50.912291 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.912146    4515 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 02 01:31:50.912291 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.912155    4515 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 02 01:31:50.915604 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.915589    4515 server.go:1263] "Started kubelet"
Nov 02 01:31:50.915697 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.915639    4515 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 02 01:31:50.915769 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.915711    4515 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 02 01:31:50.915866 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.915793    4515 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 02 01:31:50.916169 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.916095    4515 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 02 01:31:50.916159 ip-10-0-138-75 systemd[1]: Started Kubernetes Kubelet.
Nov 02 01:31:50.918842 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.918812    4515 server.go:317] "Adding debug handlers to kubelet server"
Nov 02 01:31:50.919619 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.919571    4515 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 02 01:31:50.919712 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.919670    4515 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 02 01:31:50.919780 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:50.919725    4515 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-75.us-west-1.compute.internal\" not found"
Nov 02 01:31:50.919780 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.919768    4515 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 02 01:31:50.919780 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.919777    4515 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 02 01:31:50.920043 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.920026    4515 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 02 01:31:50.920147 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.920080    4515 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 01:26:11 +0000 UTC" deadline="2028-06-05 01:58:48.087079962 +0000 UTC"
Nov 02 01:31:50.920147 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.920133    4515 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13944h26m57.166968825s"
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922473    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50bfe6af-769e-40aa-8a09-560ea1a984a2" volumeName="kubernetes.io/configmap/50bfe6af-769e-40aa-8a09-560ea1a984a2-serviceca" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922515    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0923847d-508c-4308-93f9-543d9bb3c251" volumeName="kubernetes.io/configmap/0923847d-508c-4308-93f9-543d9bb3c251-konnectivity-ca" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922532    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="340a686c-e0f6-4b71-a432-66659bae4e2d" volumeName="kubernetes.io/configmap/340a686c-e0f6-4b71-a432-66659bae4e2d-signing-cabundle" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922546    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4" volumeName="kubernetes.io/empty-dir/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-data-volume" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922560    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="414870a7-ef47-4b1e-aac2-1b25923286e9" volumeName="kubernetes.io/empty-dir/414870a7-ef47-4b1e-aac2-1b25923286e9-tmp-dir" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922567    4515 factory.go:55] Registering systemd factory
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922575    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62f7a5eb-5a3c-4bd5-a0cd-486539e66c82" volumeName="kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovnkube-config" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922585    4515 factory.go:223] Registration of the systemd container factory successfully
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922588    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e33b967e-3f0d-49c0-bae2-c95ce8341e1f" volumeName="kubernetes.io/empty-dir/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-tuned" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922603    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f40cfbd7-701b-4fb2-8756-86ae988cb1b8" volumeName="kubernetes.io/projected/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-kube-api-access-99m65" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922619    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aafee8b-ab11-4b97-afae-09b9a6d59c01" volumeName="kubernetes.io/configmap/0aafee8b-ab11-4b97-afae-09b9a6d59c01-iptables-alerter-script" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922631    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="340a686c-e0f6-4b71-a432-66659bae4e2d" volumeName="kubernetes.io/secret/340a686c-e0f6-4b71-a432-66659bae4e2d-signing-key" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922644    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db" volumeName="kubernetes.io/configmap/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-cni-binary-copy" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922660    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db" volumeName="kubernetes.io/configmap/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-daemon-config" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922673    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e33b967e-3f0d-49c0-bae2-c95ce8341e1f" volumeName="kubernetes.io/empty-dir/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-tmp" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922686    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed8d1d91-e435-42f1-a875-f87fd18e0b87" volumeName="kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922701    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50bfe6af-769e-40aa-8a09-560ea1a984a2" volumeName="kubernetes.io/projected/50bfe6af-769e-40aa-8a09-560ea1a984a2-kube-api-access-jrjqz" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922715    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1092321a-6edf-43d5-bb31-5efb0b80990e" volumeName="kubernetes.io/projected/1092321a-6edf-43d5-bb31-5efb0b80990e-kube-api-access-skmsl" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922728    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="340a686c-e0f6-4b71-a432-66659bae4e2d" volumeName="kubernetes.io/projected/340a686c-e0f6-4b71-a432-66659bae4e2d-kube-api-access-5d6xw" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922741    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8c429f46-34a2-48a4-8fe5-8204fc5ff5cb" volumeName="kubernetes.io/projected/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-kube-api-access-dhht9" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922753    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f40cfbd7-701b-4fb2-8756-86ae988cb1b8" volumeName="kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cni-binary-copy" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922766    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aafee8b-ab11-4b97-afae-09b9a6d59c01" volumeName="kubernetes.io/projected/0aafee8b-ab11-4b97-afae-09b9a6d59c01-kube-api-access-dfc7g" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922777    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4" volumeName="kubernetes.io/projected/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-api-access-r6hbj" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922789    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62f7a5eb-5a3c-4bd5-a0cd-486539e66c82" volumeName="kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovnkube-script-lib" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922801    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62f7a5eb-5a3c-4bd5-a0cd-486539e66c82" volumeName="kubernetes.io/projected/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-kube-api-access-pjz92" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922814    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d49e5be4-4550-4c23-b33c-e01c22d822fe" volumeName="kubernetes.io/projected/d49e5be4-4550-4c23-b33c-e01c22d822fe-kube-api-access-h2x8f" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922827    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd401370-c917-4608-b6df-da7367f1ad21" volumeName="kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-service-ca-bundle" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922840    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd401370-c917-4608-b6df-da7367f1ad21" volumeName="kubernetes.io/projected/fd401370-c917-4608-b6df-da7367f1ad21-kube-api-access-c6pv9" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922852    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8c429f46-34a2-48a4-8fe5-8204fc5ff5cb" volumeName="kubernetes.io/configmap/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-config-volume" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922866    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24" volumeName="kubernetes.io/projected/a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24-kube-api-access-5p9nx" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922884    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0923847d-508c-4308-93f9-543d9bb3c251" volumeName="kubernetes.io/secret/0923847d-508c-4308-93f9-543d9bb3c251-agent-certs" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922898    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62f7a5eb-5a3c-4bd5-a0cd-486539e66c82" volumeName="kubernetes.io/secret/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922911    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd401370-c917-4608-b6df-da7367f1ad21" volumeName="kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-trusted-ca-bundle" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922924    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="696e3b37-d3bb-48e4-9836-afb85f9e34fd" volumeName="kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922968    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db" volumeName="kubernetes.io/projected/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-kube-api-access-mdkwl" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922983    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd401370-c917-4608-b6df-da7367f1ad21" volumeName="kubernetes.io/empty-dir/fd401370-c917-4608-b6df-da7367f1ad21-archives-path" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.922998    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="414870a7-ef47-4b1e-aac2-1b25923286e9" volumeName="kubernetes.io/projected/414870a7-ef47-4b1e-aac2-1b25923286e9-kube-api-access-2kctw" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.923018    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5a3c8cf1-c66c-4f81-9a40-c19a26df3f77" volumeName="kubernetes.io/projected/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-kube-api-access-vsdhq" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.923035    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e33b967e-3f0d-49c0-bae2-c95ce8341e1f" volumeName="kubernetes.io/projected/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-kube-api-access-gjsm4" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.923073    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62f7a5eb-5a3c-4bd5-a0cd-486539e66c82" volumeName="kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-env-overrides" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.923086    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8c429f46-34a2-48a4-8fe5-8204fc5ff5cb" volumeName="kubernetes.io/empty-dir/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-tmp-dir" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.923093    4515 factory.go:153] Registering CRI-O factory
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.923098    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d49e5be4-4550-4c23-b33c-e01c22d822fe" volumeName="kubernetes.io/secret/d49e5be4-4550-4c23-b33c-e01c22d822fe-cert" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.923108    4515 factory.go:223] Registration of the crio container factory successfully
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.923214    4515 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.923253    4515 factory.go:103] Registering Raw factory
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.923110    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f40cfbd7-701b-4fb2-8756-86ae988cb1b8" volumeName="kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.923270    4515 manager.go:1199] Started watching for new ooms in manager
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.923292    4515 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f40cfbd7-701b-4fb2-8756-86ae988cb1b8" volumeName="kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.923312    4515 reconstruct.go:97] "Volume reconstruction finished"
Nov 02 01:31:50.923622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.923320    4515 reconciler.go:29] "Reconciler: start to sync state"
Nov 02 01:31:50.927727 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.923731    4515 manager.go:319] Starting recovery of all containers
Nov 02 01:31:50.941051 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.941029    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 01:31:50.942663 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.942642    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 01:31:50.942909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.942866    4515 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 02 01:31:50.945766 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.945747    4515 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 02 01:31:50.945766 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.945768    4515 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 02 01:31:50.945902 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.945789    4515 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 02 01:31:50.945902 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:50.945830    4515 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 02 01:31:50.955681 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.955659    4515 generic.go:356] "Generic (PLEG): container finished" podID="f40cfbd7-701b-4fb2-8756-86ae988cb1b8" containerID="31d9a0c15850681445731b663e6e32b0e5ee0c7610c88c5b0a1854fc2a7fdf0a" exitCode=0
Nov 02 01:31:50.955681 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.955679    4515 generic.go:356] "Generic (PLEG): container finished" podID="f40cfbd7-701b-4fb2-8756-86ae988cb1b8" containerID="b4380ad579458435025e7b90e9fee8d04e8d5eb4484eb61c9c2e9e8472615289" exitCode=0
Nov 02 01:31:50.955792 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.955688    4515 generic.go:356] "Generic (PLEG): container finished" podID="f40cfbd7-701b-4fb2-8756-86ae988cb1b8" containerID="8d114b8f6b4aa0deab45a07a2747f02249a004ed97e6977a880d3fc40cef44da" exitCode=0
Nov 02 01:31:50.955792 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.955695    4515 generic.go:356] "Generic (PLEG): container finished" podID="f40cfbd7-701b-4fb2-8756-86ae988cb1b8" containerID="a5613059dc0cb54ad5210892f583c26b556f3ee5b1c8e6137393a1ba9fca7ffe" exitCode=0
Nov 02 01:31:50.955792 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.955701    4515 generic.go:356] "Generic (PLEG): container finished" podID="f40cfbd7-701b-4fb2-8756-86ae988cb1b8" containerID="548e285af4a9dea0d4797a6d2c0312e5d80ae526e5977100547f20d7ee659642" exitCode=0
Nov 02 01:31:50.955792 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.955706    4515 generic.go:356] "Generic (PLEG): container finished" podID="f40cfbd7-701b-4fb2-8756-86ae988cb1b8" containerID="6e55565b1ec11f9a13534a68679da53b4b7a28a616dcf28122e08cc05233b1db" exitCode=0
Nov 02 01:31:50.961530 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.961508    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 01:31:50.961902 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.961880    4515 generic.go:356] "Generic (PLEG): container finished" podID="62f7a5eb-5a3c-4bd5-a0cd-486539e66c82" containerID="b93c320b687bf7b18eaabfa2e208861fbd03e06b6e09fd09bec13f5710983a2a" exitCode=1
Nov 02 01:31:50.982138 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.982057    4515 generic.go:356] "Generic (PLEG): container finished" podID="fe959ce17f01eeb1eaeed7edd9cd6e8b" containerID="55672280fe2ed3bf86882a8dd74104d8771e770f91b578a817bf75a2a17e7d0a" exitCode=0
Nov 02 01:31:50.994969 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:50.994954    4515 manager.go:324] Recovery completed
Nov 02 01:31:51.009056 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.009039    4515 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 01:31:51.010155 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.010139    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 01:31:51.010232 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.010168    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 01:31:51.010232 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.010179    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 01:31:51.011529 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.011512    4515 cpu_manager.go:226] "Starting" policy="none"
Nov 02 01:31:51.011529 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.011527    4515 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 02 01:31:51.011624 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.011548    4515 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 02 01:31:51.011624 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.011528    4515 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 01:31:51.011694 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.011677    4515 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 02 01:31:51.011738 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.011697    4515 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 02 01:31:51.011777 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.011738    4515 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 02 01:31:51.011777 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.011747    4515 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 02 01:31:51.011777 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.011755    4515 policy_none.go:50] "Start"
Nov 02 01:31:51.011777 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.011765    4515 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 02 01:31:51.011876 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.011778    4515 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 02 01:31:51.011953 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.011942    4515 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 02 01:31:51.011982 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.011954    4515 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 02 01:31:51.011982 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.011960    4515 policy_none.go:44] "Start"
Nov 02 01:31:51.012386 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.012371    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 01:31:51.012435 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.012399    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 01:31:51.012435 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.012410    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 01:31:51.016061 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.016034    4515 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 02 01:31:51.019249 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.019227    4515 manager.go:341] "Starting Device Plugin manager"
Nov 02 01:31:51.019249 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:51.019250    4515 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 02 01:31:51.019387 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.019258    4515 server.go:87] "Starting device plugin registration server"
Nov 02 01:31:51.019499 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.019486    4515 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 02 01:31:51.019553 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.019498    4515 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 02 01:31:51.019601 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.019569    4515 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 02 01:31:51.019693 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.019662    4515 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 02 01:31:51.019748 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.019706    4515 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 02 01:31:51.019748 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.019713    4515 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 02 01:31:51.019847 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.019763    4515 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-02T01:31:51.019691003Z","UUID":"5c123c7a-6693-47ac-8e50-202537382f8a","Handler":null,"Name":"","Endpoint":""}
Nov 02 01:31:51.020974 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.020956    4515 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 02 01:31:51.021072 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.020980    4515 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 02 01:31:51.021150 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:51.021078    4515 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 02 01:31:51.021213 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:51.021174    4515 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-75.us-west-1.compute.internal\" not found"
Nov 02 01:31:51.046551 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.046510    4515 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal"]
Nov 02 01:31:51.046648 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.046626    4515 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 01:31:51.047505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.047481    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 01:31:51.047583 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.047513    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 01:31:51.047583 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.047524    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 01:31:51.047762 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.047751    4515 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 01:31:51.047920 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.047905    4515 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 01:31:51.048529 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.048512    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 01:31:51.048529 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.048518    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 01:31:51.048635 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.048538    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 01:31:51.048635 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.048542    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 01:31:51.048635 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.048549    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 01:31:51.048635 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.048553    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 01:31:51.048760 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:51.048676    4515 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-75.us-west-1.compute.internal\" not found" node="ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.048830 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.048802    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal" event={"ID":"e1f480d0264a553be3287c8f045c192e","Type":"ContainerStarted","Data":"305772659dd5f86a6224f4132efc5f940212390b77be84465ceed587b3e88b88"}
Nov 02 01:31:51.048862 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.048836    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal" event={"ID":"e1f480d0264a553be3287c8f045c192e","Type":"ContainerStarted","Data":"d1e73f09594b5e38baccb49b8c5d3d2936ef28ad507f41fa1d8db246627b9a16"}
Nov 02 01:31:51.048889 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.048864    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal" event={"ID":"fe959ce17f01eeb1eaeed7edd9cd6e8b","Type":"ContainerStarted","Data":"2a17dde8335fab2e71fc4ecfbdc2ff2e747dcd16fc47c8cea17574e9d6232715"}
Nov 02 01:31:51.048889 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.048877    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal" event={"ID":"fe959ce17f01eeb1eaeed7edd9cd6e8b","Type":"ContainerDied","Data":"55672280fe2ed3bf86882a8dd74104d8771e770f91b578a817bf75a2a17e7d0a"}
Nov 02 01:31:51.048889 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.048881    4515 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 01:31:51.048986 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.048889    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal" event={"ID":"fe959ce17f01eeb1eaeed7edd9cd6e8b","Type":"ContainerStarted","Data":"e896a197f81fdba8c0f0e3a2b999f02eecbcfc614349e74b00e3c79f2fec99aa"}
Nov 02 01:31:51.049565 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.049551    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 01:31:51.049607 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.049576    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 01:31:51.049607 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.049585    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 01:31:51.049699 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:51.049688    4515 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-75.us-west-1.compute.internal\" not found" node="ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.075500 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.075476    4515 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 02 01:31:51.077742 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.077727    4515 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 01:31:51.077812 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.077742    4515 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 01:31:51.078397 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.078381    4515 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 01:31:51.119980 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.119955    4515 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 01:31:51.120217 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.120202    4515 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.120804 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.120788    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 01:31:51.120887 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.120820    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 01:31:51.120887 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.120836    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 01:31:51.120887 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.120877    4515 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.128372 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.128351    4515 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 01:31:51.176487 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.176461    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe959ce17f01eeb1eaeed7edd9cd6e8b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"fe959ce17f01eeb1eaeed7edd9cd6e8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.176629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.176491    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe959ce17f01eeb1eaeed7edd9cd6e8b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"fe959ce17f01eeb1eaeed7edd9cd6e8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.176629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.176510    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e1f480d0264a553be3287c8f045c192e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"e1f480d0264a553be3287c8f045c192e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.180209 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.180196    4515 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 01:31:51.180260 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:51.180234    4515 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.180260 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.180249    4515 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.233229 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.233174    4515 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.233307 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.233238    4515 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.233758 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.233744    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeNotReady"
Nov 02 01:31:51.233842 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.233774    4515 setters.go:546] "Node became not ready" node="ip-10-0-138-75.us-west-1.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-02T01:31:51Z","lastTransitionTime":"2026-11-02T01:31:51Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Nov 02 01:31:51.238266 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.238251    4515 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 01:31:51.238309 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:51.238286    4515 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.241462 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.241449    4515 nodeinfomanager.go:471] existing CSINode "ip-10-0-138-75.us-west-1.compute.internal" is owned by different node (oldNodeID="b397ccf6-1d17-455b-a784-d33c5f50597d", newNodeID=""), cleaning up...
Nov 02 01:31:51.276638 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.276614    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe959ce17f01eeb1eaeed7edd9cd6e8b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"fe959ce17f01eeb1eaeed7edd9cd6e8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.276725 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.276640    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe959ce17f01eeb1eaeed7edd9cd6e8b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"fe959ce17f01eeb1eaeed7edd9cd6e8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.276725 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.276657    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e1f480d0264a553be3287c8f045c192e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"e1f480d0264a553be3287c8f045c192e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.276725 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.276698    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e1f480d0264a553be3287c8f045c192e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"e1f480d0264a553be3287c8f045c192e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.276725 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.276716    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe959ce17f01eeb1eaeed7edd9cd6e8b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"fe959ce17f01eeb1eaeed7edd9cd6e8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.276849 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.276727    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe959ce17f01eeb1eaeed7edd9cd6e8b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal\" (UID: \"fe959ce17f01eeb1eaeed7edd9cd6e8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.866971 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.866939    4515 apiserver.go:52] "Watching apiserver"
Nov 02 01:31:51.986531 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.986504    4515 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:51.986669 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:51.986649    4515 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:52.001016 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.000992    4515 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 02 01:31:52.001560 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.001540    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-cqspb","openshift-insights/periodic-gathering-hmn9t-cwkdz","openshift-multus/multus-additional-cni-plugins-9k54v","openshift-cluster-node-tuning-operator/tuned-47cxd","openshift-dns/node-resolver-cfv7t","kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal","openshift-service-ca/service-ca-5fffc97bdf-rrqpk","openshift-insights/insights-runtime-extractor-cnkn6","openshift-network-diagnostics/network-check-target-nth82","openshift-multus/network-metrics-daemon-p75zz","openshift-network-operator/iptables-alerter-9bccs","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52","openshift-ingress-canary/ingress-canary-lz5s9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal","openshift-multus/multus-f6pjf","openshift-image-registry/node-ca-z5pbt","openshift-ovn-kubernetes/ovnkube-node-gs99s","openshift-dns/dns-default-dcc98","openshift-kube-storage-version-migrator/migrator-66df5d496b-2r2kf","kube-system/konnectivity-agent-hbcss"]
Nov 02 01:31:52.004544 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.004524    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:52.009919 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.009897    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:52.010331 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.010313    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:52.021611 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.021594    4515 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 02 01:31:52.046235 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.046213    4515 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 01:31:52.046309 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:52.046258    4515 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:52.046947 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.046935    4515 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 01:31:52.046977 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:52.046971    4515 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-75.us-west-1.compute.internal"
Nov 02 01:31:52.054784 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.054765    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 02 01:31:52.054861 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.054782    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 02 01:31:52.054861 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.054782    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 02 01:31:52.055161 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.055144    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 02 01:31:52.055245 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.055230    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 02 01:31:52.055363 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.055330    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-n6fsn\""
Nov 02 01:31:52.055676 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.055660    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-kk5lr\""
Nov 02 01:31:52.055822 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.055807    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j52qq\""
Nov 02 01:31:52.055878 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.055808    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 02 01:31:52.055878 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.055813    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 02 01:31:52.055878 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.055844    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 02 01:31:52.056856 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.056840    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-z6qvz\""
Nov 02 01:31:52.081047 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081024    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-kubelet\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.081156 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081052    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/340a686c-e0f6-4b71-a432-66659bae4e2d-signing-cabundle\") pod \"service-ca-5fffc97bdf-rrqpk\" (UID: \"340a686c-e0f6-4b71-a432-66659bae4e2d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk"
Nov 02 01:31:52.081156 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081070    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-tuned\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.081156 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081085    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhht9\" (UniqueName: \"kubernetes.io/projected/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-kube-api-access-dhht9\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:52.081156 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081102    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-kubelet\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.081156 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081135    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-env-overrides\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.081395 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081199    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:52.081395 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081238    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysctl-d\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.081395 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081281    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-tuned\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.081395 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081353    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-run\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.081395 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081375    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:52.081602 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081403    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-conf-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.081602 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081428    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-systemd-units\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.081602 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081449    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-var-lib-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.081602 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081471    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-sys-fs\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.081602 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081504    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:52.081602 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081532    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6hbj\" (UniqueName: \"kubernetes.io/projected/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-api-access-r6hbj\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:52.081602 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081568    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-cni-bin\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.081788 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081634    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-systemd\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.081788 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081661    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-etc-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.081788 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081683    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-node-log\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.081788 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081708    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjz92\" (UniqueName: \"kubernetes.io/projected/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-kube-api-access-pjz92\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.081903 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081787    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0aafee8b-ab11-4b97-afae-09b9a6d59c01-iptables-alerter-script\") pod \"iptables-alerter-9bccs\" (UID: \"0aafee8b-ab11-4b97-afae-09b9a6d59c01\") " pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:52.081903 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081830    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:52.081903 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081888    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-tmp-dir\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:52.082037 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081926    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-cni-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.082037 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.081985    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-tmp-dir\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:52.082037 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082013    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d6xw\" (UniqueName: \"kubernetes.io/projected/340a686c-e0f6-4b71-a432-66659bae4e2d-kube-api-access-5d6xw\") pod \"service-ca-5fffc97bdf-rrqpk\" (UID: \"340a686c-e0f6-4b71-a432-66659bae4e2d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk"
Nov 02 01:31:52.082189 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082151    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-run-ovn-kubernetes\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.082189 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082183    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsdhq\" (UniqueName: \"kubernetes.io/projected/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-kube-api-access-vsdhq\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.082280 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082199    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-crio-socket\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:52.082280 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082215    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d49e5be4-4550-4c23-b33c-e01c22d822fe-cert\") pod \"ingress-canary-lz5s9\" (UID: \"d49e5be4-4550-4c23-b33c-e01c22d822fe\") " pod="openshift-ingress-canary/ingress-canary-lz5s9"
Nov 02 01:31:52.082371 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082320    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/340a686c-e0f6-4b71-a432-66659bae4e2d-signing-key\") pod \"service-ca-5fffc97bdf-rrqpk\" (UID: \"340a686c-e0f6-4b71-a432-66659bae4e2d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk"
Nov 02 01:31:52.082371 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082356    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovnkube-config\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.082463 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082382    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovn-node-metrics-cert\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.082463 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082408    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kctw\" (UniqueName: \"kubernetes.io/projected/414870a7-ef47-4b1e-aac2-1b25923286e9-kube-api-access-2kctw\") pod \"node-resolver-cfv7t\" (UID: \"414870a7-ef47-4b1e-aac2-1b25923286e9\") " pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:52.082463 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082433    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysctl-conf\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.082599 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082486    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-cni-multus\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.082599 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082514    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-service-ca-bundle\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:52.082599 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082544    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:52.082599 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082565    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99m65\" (UniqueName: \"kubernetes.io/projected/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-kube-api-access-99m65\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:52.082781 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082625    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0923847d-508c-4308-93f9-543d9bb3c251-konnectivity-ca\") pod \"konnectivity-agent-hbcss\" (UID: \"0923847d-508c-4308-93f9-543d9bb3c251\") " pod="kube-system/konnectivity-agent-hbcss"
Nov 02 01:31:52.082781 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082655    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-lib-modules\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.082781 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082703    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-config-volume\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:52.082781 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082731    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-etc-kubernetes\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.082781 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082756    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdkwl\" (UniqueName: \"kubernetes.io/projected/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-kube-api-access-mdkwl\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.083012 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082784    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/696e3b37-d3bb-48e4-9836-afb85f9e34fd-kubelet-config\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:52.083012 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082808    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-socket-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.083012 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082833    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cni-binary-copy\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:52.083012 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082864    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2x8f\" (UniqueName: \"kubernetes.io/projected/d49e5be4-4550-4c23-b33c-e01c22d822fe-kube-api-access-h2x8f\") pod \"ingress-canary-lz5s9\" (UID: \"d49e5be4-4550-4c23-b33c-e01c22d822fe\") " pod="openshift-ingress-canary/ingress-canary-lz5s9"
Nov 02 01:31:52.083012 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082882    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrjqz\" (UniqueName: \"kubernetes.io/projected/50bfe6af-769e-40aa-8a09-560ea1a984a2-kube-api-access-jrjqz\") pod \"node-ca-z5pbt\" (UID: \"50bfe6af-769e-40aa-8a09-560ea1a984a2\") " pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:52.083012 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082919    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-trusted-ca-bundle\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:52.083012 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082948    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p9nx\" (UniqueName: \"kubernetes.io/projected/a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24-kube-api-access-5p9nx\") pod \"migrator-66df5d496b-2r2kf\" (UID: \"a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-2r2kf"
Nov 02 01:31:52.083012 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.082974    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rws7g\" (UniqueName: \"kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g\") pod \"network-check-target-nth82\" (UID: \"ed8d1d91-e435-42f1-a875-f87fd18e0b87\") " pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:52.083012 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083001    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-modprobe-d\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.083418 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083026    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-sys\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.083418 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083032    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0923847d-508c-4308-93f9-543d9bb3c251-konnectivity-ca\") pod \"konnectivity-agent-hbcss\" (UID: \"0923847d-508c-4308-93f9-543d9bb3c251\") " pod="kube-system/konnectivity-agent-hbcss"
Nov 02 01:31:52.083418 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083071    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/50bfe6af-769e-40aa-8a09-560ea1a984a2-serviceca\") pod \"node-ca-z5pbt\" (UID: \"50bfe6af-769e-40aa-8a09-560ea1a984a2\") " pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:52.083418 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083104    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-ovn\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.083418 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083162    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.083418 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083217    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfc7g\" (UniqueName: \"kubernetes.io/projected/0aafee8b-ab11-4b97-afae-09b9a6d59c01-kube-api-access-dfc7g\") pod \"iptables-alerter-9bccs\" (UID: \"0aafee8b-ab11-4b97-afae-09b9a6d59c01\") " pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:52.083418 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083247    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0923847d-508c-4308-93f9-543d9bb3c251-agent-certs\") pod \"konnectivity-agent-hbcss\" (UID: \"0923847d-508c-4308-93f9-543d9bb3c251\") " pod="kube-system/konnectivity-agent-hbcss"
Nov 02 01:31:52.083418 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083271    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:52.083418 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083298    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-registration-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.083418 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083324    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-device-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.083418 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083350    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-etc-selinux\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.083418 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083363    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/50bfe6af-769e-40aa-8a09-560ea1a984a2-serviceca\") pod \"node-ca-z5pbt\" (UID: \"50bfe6af-769e-40aa-8a09-560ea1a984a2\") " pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:52.083418 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083373    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/414870a7-ef47-4b1e-aac2-1b25923286e9-hosts-file\") pod \"node-resolver-cfv7t\" (UID: \"414870a7-ef47-4b1e-aac2-1b25923286e9\") " pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:52.083418 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083406    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-kubernetes\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.083860 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083444    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-systemd\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.083860 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083469    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-host\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.083860 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083495    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-tmp\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.083860 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083523    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0923847d-508c-4308-93f9-543d9bb3c251-agent-certs\") pod \"konnectivity-agent-hbcss\" (UID: \"0923847d-508c-4308-93f9-543d9bb3c251\") " pod="kube-system/konnectivity-agent-hbcss"
Nov 02 01:31:52.083860 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083540    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cnibin\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:52.083860 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083564    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-tmp\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.083860 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083595    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:52.083860 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083613    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysconfig\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.083860 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083671    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-socket-dir-parent\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.083860 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083703    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-k8s-cni-cncf-io\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.083860 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083742    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-daemon-config\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.083860 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083762    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skmsl\" (UniqueName: \"kubernetes.io/projected/1092321a-6edf-43d5-bb31-5efb0b80990e-kube-api-access-skmsl\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:52.083860 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083778    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-var-lib-kubelet\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.083860 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083793    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-multus-certs\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.084321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083892    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50bfe6af-769e-40aa-8a09-560ea1a984a2-host\") pod \"node-ca-z5pbt\" (UID: \"50bfe6af-769e-40aa-8a09-560ea1a984a2\") " pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:52.084321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083910    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-cni-netd\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.084321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083926    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovnkube-script-lib\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.084321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083941    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fd401370-c917-4608-b6df-da7367f1ad21-archives-path\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:52.084321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.083969    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6pv9\" (UniqueName: \"kubernetes.io/projected/fd401370-c917-4608-b6df-da7367f1ad21-kube-api-access-c6pv9\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:52.084321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084000    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/414870a7-ef47-4b1e-aac2-1b25923286e9-tmp-dir\") pod \"node-resolver-cfv7t\" (UID: \"414870a7-ef47-4b1e-aac2-1b25923286e9\") " pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:52.084321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084003    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fd401370-c917-4608-b6df-da7367f1ad21-archives-path\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:52.084321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084082    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0aafee8b-ab11-4b97-afae-09b9a6d59c01-host-slash\") pod \"iptables-alerter-9bccs\" (UID: \"0aafee8b-ab11-4b97-afae-09b9a6d59c01\") " pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:52.084321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084131    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/414870a7-ef47-4b1e-aac2-1b25923286e9-tmp-dir\") pod \"node-resolver-cfv7t\" (UID: \"414870a7-ef47-4b1e-aac2-1b25923286e9\") " pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:52.084321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084144    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-data-volume\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:52.084321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084205    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-data-volume\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:52.084321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084226    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/696e3b37-d3bb-48e4-9836-afb85f9e34fd-dbus\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:52.084321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084244    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-slash\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.084321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084270    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-run-netns\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.084321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084287    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-cni-bin\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.084838 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084325    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.084838 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084352    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjsm4\" (UniqueName: \"kubernetes.io/projected/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-kube-api-access-gjsm4\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.084838 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084382    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-hostroot\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.084838 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084409    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.084838 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084425    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-log-socket\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.084838 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084459    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-system-cni-dir\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:52.084838 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084480    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:52.084838 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084517    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-system-cni-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.084838 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084563    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-cnibin\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.084838 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084603    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-cni-binary-copy\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.084838 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.084631    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-netns\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.140963 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.140879    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsdhq\" (UniqueName: \"kubernetes.io/projected/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-kube-api-access-vsdhq\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.146066 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.146042    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrjqz\" (UniqueName: \"kubernetes.io/projected/50bfe6af-769e-40aa-8a09-560ea1a984a2-kube-api-access-jrjqz\") pod \"node-ca-z5pbt\" (UID: \"50bfe6af-769e-40aa-8a09-560ea1a984a2\") " pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:52.155066 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155047    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 02 01:31:52.155066 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155060    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 02 01:31:52.155204 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155081    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 02 01:31:52.155204 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155177    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 02 01:31:52.155297 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155282    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 02 01:31:52.155345 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155289    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 02 01:31:52.155394 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155354    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 02 01:31:52.155394 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155376    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 02 01:31:52.155494 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155402    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 02 01:31:52.155494 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155430    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 02 01:31:52.155683 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155584    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 02 01:31:52.155683 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155609    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-846kb\""
Nov 02 01:31:52.155683 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155629    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 02 01:31:52.155799 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155685    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 02 01:31:52.155799 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155755    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 02 01:31:52.155799 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155763    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 02 01:31:52.155911 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155829    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 02 01:31:52.155911 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155846    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 02 01:31:52.155911 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155895    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 02 01:31:52.156008 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.155927    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 02 01:31:52.156100 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.156080    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 02 01:31:52.156170 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.156090    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 02 01:31:52.156212 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.156201    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 02 01:31:52.156367 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.156349    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 02 01:31:52.156499 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.156485    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 02 01:31:52.156555 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.156501    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 02 01:31:52.156555 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.156489    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 02 01:31:52.156555 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.156529    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 02 01:31:52.156555 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.156554    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 02 01:31:52.156712 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.156551    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 02 01:31:52.156712 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.156589    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 02 01:31:52.156712 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.156514    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skmsl\" (UniqueName: \"kubernetes.io/projected/1092321a-6edf-43d5-bb31-5efb0b80990e-kube-api-access-skmsl\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:52.156712 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.156626    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 02 01:31:52.156712 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.156645    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xv9qq\""
Nov 02 01:31:52.156712 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.156691    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fxpjk\""
Nov 02 01:31:52.158197 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.158181    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 02 01:31:52.158242 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.158198    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 02 01:31:52.158242 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.158207    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 02 01:31:52.160341 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.160320    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wfswr\""
Nov 02 01:31:52.160692 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.160658    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 02 01:31:52.160767 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.160732    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjz92\" (UniqueName: \"kubernetes.io/projected/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-kube-api-access-pjz92\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.160767 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.160740    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhht9\" (UniqueName: \"kubernetes.io/projected/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-kube-api-access-dhht9\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:52.160856 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.160838    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6hbj\" (UniqueName: \"kubernetes.io/projected/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-api-access-r6hbj\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:52.160888 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.160850    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kctw\" (UniqueName: \"kubernetes.io/projected/414870a7-ef47-4b1e-aac2-1b25923286e9-kube-api-access-2kctw\") pod \"node-resolver-cfv7t\" (UID: \"414870a7-ef47-4b1e-aac2-1b25923286e9\") " pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:52.160924 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.160910    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d6xw\" (UniqueName: \"kubernetes.io/projected/340a686c-e0f6-4b71-a432-66659bae4e2d-kube-api-access-5d6xw\") pod \"service-ca-5fffc97bdf-rrqpk\" (UID: \"340a686c-e0f6-4b71-a432-66659bae4e2d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk"
Nov 02 01:31:52.160963 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.160947    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdkwl\" (UniqueName: \"kubernetes.io/projected/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-kube-api-access-mdkwl\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.161187 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.161173    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99m65\" (UniqueName: \"kubernetes.io/projected/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-kube-api-access-99m65\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:52.161306 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.161294    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-env-overrides\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.161415 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.161403    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/340a686c-e0f6-4b71-a432-66659bae4e2d-signing-cabundle\") pod \"service-ca-5fffc97bdf-rrqpk\" (UID: \"340a686c-e0f6-4b71-a432-66659bae4e2d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk"
Nov 02 01:31:52.161530 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.161518    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:52.162238 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.162177    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0aafee8b-ab11-4b97-afae-09b9a6d59c01-iptables-alerter-script\") pod \"iptables-alerter-9bccs\" (UID: \"0aafee8b-ab11-4b97-afae-09b9a6d59c01\") " pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:52.162480 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.162466    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2x8f\" (UniqueName: \"kubernetes.io/projected/d49e5be4-4550-4c23-b33c-e01c22d822fe-kube-api-access-h2x8f\") pod \"ingress-canary-lz5s9\" (UID: \"d49e5be4-4550-4c23-b33c-e01c22d822fe\") " pod="openshift-ingress-canary/ingress-canary-lz5s9"
Nov 02 01:31:52.162691 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.162653    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovnkube-config\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.162721 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.162675    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/340a686c-e0f6-4b71-a432-66659bae4e2d-signing-key\") pod \"service-ca-5fffc97bdf-rrqpk\" (UID: \"340a686c-e0f6-4b71-a432-66659bae4e2d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk"
Nov 02 01:31:52.162721 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.162701    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d49e5be4-4550-4c23-b33c-e01c22d822fe-cert\") pod \"ingress-canary-lz5s9\" (UID: \"d49e5be4-4550-4c23-b33c-e01c22d822fe\") " pod="openshift-ingress-canary/ingress-canary-lz5s9"
Nov 02 01:31:52.162721 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.162672    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-service-ca-bundle\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:52.162806 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.162767    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovn-node-metrics-cert\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.162902 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.162889    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-config-volume\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:52.163017 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.163005    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cni-binary-copy\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:52.164178 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.164165    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-daemon-config\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.164217 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.164166    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:52.165316 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.165301    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-cni-binary-copy\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.166744 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.166730    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfc7g\" (UniqueName: \"kubernetes.io/projected/0aafee8b-ab11-4b97-afae-09b9a6d59c01-kube-api-access-dfc7g\") pod \"iptables-alerter-9bccs\" (UID: \"0aafee8b-ab11-4b97-afae-09b9a6d59c01\") " pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:52.166856 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.166836    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjsm4\" (UniqueName: \"kubernetes.io/projected/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-kube-api-access-gjsm4\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.166890 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.166854    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rws7g\" (UniqueName: \"kubernetes.io/projected/ed8d1d91-e435-42f1-a875-f87fd18e0b87-kube-api-access-rws7g\") pod \"network-check-target-nth82\" (UID: \"ed8d1d91-e435-42f1-a875-f87fd18e0b87\") " pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:52.166993 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.166978    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6pv9\" (UniqueName: \"kubernetes.io/projected/fd401370-c917-4608-b6df-da7367f1ad21-kube-api-access-c6pv9\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:52.185195 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185177    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:52.185259 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185205    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-system-cni-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.185259 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185222    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-cnibin\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.185325 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185282    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-cnibin\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.185325 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185291    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-system-cni-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.185325 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:52.185320    4515 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 01:31:52.185414 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185335    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-netns\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.185414 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185365    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-kubelet\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.185414 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:52.185387    4515 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs podName:1092321a-6edf-43d5-bb31-5efb0b80990e nodeName:}" failed. No retries permitted until 2026-11-02 01:31:52.685372045 +0000 UTC m=+1.897741856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs") pod "network-metrics-daemon-p75zz" (UID: "1092321a-6edf-43d5-bb31-5efb0b80990e") : secret "metrics-daemon-secret" not found
Nov 02 01:31:52.185414 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185395    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-kubelet\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.185414 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185408    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-kubelet\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.185414 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185409    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-netns\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.185573 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185430    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysctl-d\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.185573 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185458    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-run\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.185573 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185480    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:52.185573 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185458    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-kubelet\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.185573 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185500    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-conf-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.185573 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185507    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-run\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.185573 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185521    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-systemd-units\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.185573 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185534    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-conf-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.185573 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185542    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-var-lib-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.185573 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:52.185548    4515 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 01:31:52.185573 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185557    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-systemd-units\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.185573 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185566    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-sys-fs\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.185573 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185572    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysctl-d\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185581    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-var-lib-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:52.185593    4515 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls podName:8c429f46-34a2-48a4-8fe5-8204fc5ff5cb nodeName:}" failed. No retries permitted until 2026-11-02 01:31:52.685582267 +0000 UTC m=+1.897952096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls") pod "dns-default-dcc98" (UID: "8c429f46-34a2-48a4-8fe5-8204fc5ff5cb") : secret "dns-default-metrics-tls" not found
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185610    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185630    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-sys-fs\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185673    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-cni-bin\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185697    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-cni-bin\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185723    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-systemd\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:52.185737    4515 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185747    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-etc-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185763    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-systemd\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185769    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-node-log\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:52.185779    4515 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls podName:3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:52.685770376 +0000 UTC m=+1.898140167 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cnkn6" (UID: "3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4") : secret "insights-runtime-extractor-tls" not found
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185800    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-node-log\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185809    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185819    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-etc-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185844    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-cni-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185870    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-run-ovn-kubernetes\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185885    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-crio-socket\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:52.185901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185903    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-cni-dir\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185918    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysctl-conf\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185938    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-run-ovn-kubernetes\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185952    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-crio-socket\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185954    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-cni-multus\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.185993    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186012    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-var-lib-cni-multus\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186013    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-lib-modules\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186043    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-etc-kubernetes\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186064    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/696e3b37-d3bb-48e4-9836-afb85f9e34fd-kubelet-config\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186062    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysctl-conf\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186074    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186093    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-lib-modules\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186094    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-etc-kubernetes\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186104    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/696e3b37-d3bb-48e4-9836-afb85f9e34fd-kubelet-config\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186136    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-socket-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186156    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-socket-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186157    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-modprobe-d\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186180    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-sys\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186196    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-ovn\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186210    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186226    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-registration-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186225    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-sys\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186240    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-device-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186230    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-modprobe-d\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186268    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-ovn\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186273    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-etc-selinux\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186304    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/414870a7-ef47-4b1e-aac2-1b25923286e9-hosts-file\") pod \"node-resolver-cfv7t\" (UID: \"414870a7-ef47-4b1e-aac2-1b25923286e9\") " pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186311    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-registration-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186320    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-kubernetes\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186322    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-device-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186308    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186334    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-systemd\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186350    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5a3c8cf1-c66c-4f81-9a40-c19a26df3f77-etc-selinux\") pod \"aws-ebs-csi-driver-node-6rf52\" (UID: \"5a3c8cf1-c66c-4f81-9a40-c19a26df3f77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rf52"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186355    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-host\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186359    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/414870a7-ef47-4b1e-aac2-1b25923286e9-hosts-file\") pod \"node-resolver-cfv7t\" (UID: \"414870a7-ef47-4b1e-aac2-1b25923286e9\") " pod="openshift-dns/node-resolver-cfv7t"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186387    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cnibin\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186403    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-systemd\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186418    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-kubernetes\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186419    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-host\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186450    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysconfig\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186486    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-cnibin\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186491    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-etc-sysconfig\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.186505 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186519    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-socket-dir-parent\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186534    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-k8s-cni-cncf-io\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186549    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-var-lib-kubelet\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186561    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-multus-socket-dir-parent\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186566    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-multus-certs\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186586    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-multus-certs\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186587    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50bfe6af-769e-40aa-8a09-560ea1a984a2-host\") pod \"node-ca-z5pbt\" (UID: \"50bfe6af-769e-40aa-8a09-560ea1a984a2\") " pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186603    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-cni-netd\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186607    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-host-run-k8s-cni-cncf-io\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186627    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0aafee8b-ab11-4b97-afae-09b9a6d59c01-host-slash\") pod \"iptables-alerter-9bccs\" (UID: \"0aafee8b-ab11-4b97-afae-09b9a6d59c01\") " pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186632    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e33b967e-3f0d-49c0-bae2-c95ce8341e1f-var-lib-kubelet\") pod \"tuned-47cxd\" (UID: \"e33b967e-3f0d-49c0-bae2-c95ce8341e1f\") " pod="openshift-cluster-node-tuning-operator/tuned-47cxd"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186636    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50bfe6af-769e-40aa-8a09-560ea1a984a2-host\") pod \"node-ca-z5pbt\" (UID: \"50bfe6af-769e-40aa-8a09-560ea1a984a2\") " pod="openshift-image-registry/node-ca-z5pbt"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186643    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/696e3b37-d3bb-48e4-9836-afb85f9e34fd-dbus\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186675    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-slash\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186696    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-cni-netd\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186701    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-run-netns\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186714    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0aafee8b-ab11-4b97-afae-09b9a6d59c01-host-slash\") pod \"iptables-alerter-9bccs\" (UID: \"0aafee8b-ab11-4b97-afae-09b9a6d59c01\") " pod="openshift-network-operator/iptables-alerter-9bccs"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186731    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/696e3b37-d3bb-48e4-9836-afb85f9e34fd-dbus\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186734    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-slash\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186763    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-cni-bin\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186778    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186815    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-run-netns\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186834    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186837    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-host-cni-bin\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186857    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-hostroot\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186882    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186925    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-log-socket\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186955    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-run-openvswitch\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186959    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db-hostroot\") pod \"multus-f6pjf\" (UID: \"d0d7fa8f-7d4f-4ef5-a35d-1f3a132cc9db\") " pod="openshift-multus/multus-f6pjf"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.186984    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-system-cni-dir\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.187032    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-log-socket\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.187629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.187056    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f40cfbd7-701b-4fb2-8756-86ae988cb1b8-system-cni-dir\") pod \"multus-additional-cni-plugins-9k54v\" (UID: \"f40cfbd7-701b-4fb2-8756-86ae988cb1b8\") " pod="openshift-multus/multus-additional-cni-plugins-9k54v"
Nov 02 01:31:52.190103 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.190062    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-8qw44\""
Nov 02 01:31:52.190287 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.190186    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-m64v5\""
Nov 02 01:31:52.190287 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.190207    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 02 01:31:52.190287 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.190220    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 02 01:31:52.190287 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.190226    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-jrhl2\""
Nov 02 01:31:52.190486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.190336    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xbzw4\""
Nov 02 01:31:52.190486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.190344    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-vt6x2\""
Nov 02 01:31:52.190486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.190363    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qn9km\""
Nov 02 01:31:52.190486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.190372    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-42sf2\""
Nov 02 01:31:52.190486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.190403    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-xjcn9\""
Nov 02 01:31:52.190486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.190406    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6csd8\""
Nov 02 01:31:52.190743 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.190731    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-87db6\""
Nov 02 01:31:52.191541 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.191526    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 02 01:31:52.191541 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.191537    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 02 01:31:52.193842 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.193828    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/696e3b37-d3bb-48e4-9836-afb85f9e34fd-original-pull-secret\") pod \"global-pull-secret-syncer-cqspb\" (UID: \"696e3b37-d3bb-48e4-9836-afb85f9e34fd\") " pod="kube-system/global-pull-secret-syncer-cqspb"
Nov 02 01:31:52.195009 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.194994    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/62f7a5eb-5a3c-4bd5-a0cd-486539e66c82-ovnkube-script-lib\") pod \"ovnkube-node-gs99s\" (UID: \"62f7a5eb-5a3c-4bd5-a0cd-486539e66c82\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:52.195938 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.195922    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p9nx\" (UniqueName: \"kubernetes.io/projected/a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24-kube-api-access-5p9nx\") pod \"migrator-66df5d496b-2r2kf\" (UID: \"a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-2r2kf"
Nov 02 01:31:52.196316 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.196296    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:52.208703 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.208029    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 02 01:31:52.213621 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.213604    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-trusted-ca-bundle\") pod \"periodic-gathering-hmn9t-cwkdz\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") " pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:31:52.313004 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.312983    4515 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 01:31:52.691371 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.691333    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:52.691534 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.691412    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:52.691534 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:52.691443    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:52.691534 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:52.691484    4515 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 01:31:52.691697 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:52.691535    4515 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 01:31:52.691697 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:52.691545    4515 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls podName:3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:53.691531156 +0000 UTC m=+2.903900957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cnkn6" (UID: "3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4") : secret "insights-runtime-extractor-tls" not found
Nov 02 01:31:52.691697 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:52.691582    4515 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls podName:8c429f46-34a2-48a4-8fe5-8204fc5ff5cb nodeName:}" failed. No retries permitted until 2026-11-02 01:31:53.69156955 +0000 UTC m=+2.903939350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls") pod "dns-default-dcc98" (UID: "8c429f46-34a2-48a4-8fe5-8204fc5ff5cb") : secret "dns-default-metrics-tls" not found
Nov 02 01:31:52.691697 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:52.691535    4515 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 01:31:52.691697 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:52.691632    4515 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs podName:1092321a-6edf-43d5-bb31-5efb0b80990e nodeName:}" failed. No retries permitted until 2026-11-02 01:31:53.691622006 +0000 UTC m=+2.903991806 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs") pod "network-metrics-daemon-p75zz" (UID: "1092321a-6edf-43d5-bb31-5efb0b80990e") : secret "metrics-daemon-secret" not found
Nov 02 01:31:53.698890 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:53.698857    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:53.699308 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:53.698898    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:53.699308 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:53.698923    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:53.699308 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:53.699047    4515 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 01:31:53.699308 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:53.699065    4515 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 01:31:53.699308 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:53.699070    4515 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 01:31:53.699308 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:53.699151    4515 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls podName:8c429f46-34a2-48a4-8fe5-8204fc5ff5cb nodeName:}" failed. No retries permitted until 2026-11-02 01:31:55.699111026 +0000 UTC m=+4.911480837 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls") pod "dns-default-dcc98" (UID: "8c429f46-34a2-48a4-8fe5-8204fc5ff5cb") : secret "dns-default-metrics-tls" not found
Nov 02 01:31:53.699308 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:53.699194    4515 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls podName:3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:55.69918311 +0000 UTC m=+4.911552900 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cnkn6" (UID: "3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4") : secret "insights-runtime-extractor-tls" not found
Nov 02 01:31:53.699308 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:53.699212    4515 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs podName:1092321a-6edf-43d5-bb31-5efb0b80990e nodeName:}" failed. No retries permitted until 2026-11-02 01:31:55.699203609 +0000 UTC m=+4.911573395 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs") pod "network-metrics-daemon-p75zz" (UID: "1092321a-6edf-43d5-bb31-5efb0b80990e") : secret "metrics-daemon-secret" not found
Nov 02 01:31:53.714226 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:53.714201    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cfv7t_414870a7-ef47-4b1e-aac2-1b25923286e9/dns-node-resolver/0.log"
Nov 02 01:31:53.993190 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:53.993089    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-rrqpk" event={"ID":"340a686c-e0f6-4b71-a432-66659bae4e2d","Type":"ContainerStarted","Data":"f5404d29b1053c055650c8f66dfe6fd28846ed530bf629a9fa2122e55d4783af"}
Nov 02 01:31:54.914649 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:54.914622    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z5pbt_50bfe6af-769e-40aa-8a09-560ea1a984a2/node-ca/0.log"
Nov 02 01:31:55.121324 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:55.121292    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:55.514732 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:55.514703    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lz5s9_d49e5be4-4550-4c23-b33c-e01c22d822fe/serve-healthcheck-canary/0.log"
Nov 02 01:31:55.714492 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:55.714454    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:55.714492 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:55.714497    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:55.714783 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:55.714517    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:55.714783 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:55.714618    4515 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 01:31:55.714783 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:55.714624    4515 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 01:31:55.714783 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:55.714671    4515 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls podName:3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4 nodeName:}" failed. No retries permitted until 2026-11-02 01:31:59.714659379 +0000 UTC m=+8.927029177 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cnkn6" (UID: "3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4") : secret "insights-runtime-extractor-tls" not found
Nov 02 01:31:55.714783 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:31:55.714692    4515 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls podName:8c429f46-34a2-48a4-8fe5-8204fc5ff5cb nodeName:}" failed. No retries permitted until 2026-11-02 01:31:59.714677652 +0000 UTC m=+8.927047451 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls") pod "dns-default-dcc98" (UID: "8c429f46-34a2-48a4-8fe5-8204fc5ff5cb") : secret "dns-default-metrics-tls" not found
Nov 02 01:31:55.715051 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:55.714933    4515 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 02 01:31:55.717071 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:55.717048    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1092321a-6edf-43d5-bb31-5efb0b80990e-metrics-certs\") pod \"network-metrics-daemon-p75zz\" (UID: \"1092321a-6edf-43d5-bb31-5efb0b80990e\") " pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:55.744895 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:55.744871    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-2r2kf_a00f9f5b-4aa6-4493-afb5-dd0cd6a5ac24/migrator/0.log"
Nov 02 01:31:55.904911 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:55.904829    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p75zz"
Nov 02 01:31:56.285540 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:56.285506    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p75zz"]
Nov 02 01:31:56.289368 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:31:56.289337    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1092321a_6edf_43d5_bb31_5efb0b80990e.slice/crio-bdad564b5214dd0e075ebdfbad2026feb1e817d73634f68fc430fe432ec43496 WatchSource:0}: Error finding container bdad564b5214dd0e075ebdfbad2026feb1e817d73634f68fc430fe432ec43496: Status 404 returned error can't find the container with id bdad564b5214dd0e075ebdfbad2026feb1e817d73634f68fc430fe432ec43496
Nov 02 01:31:57.001070 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:57.001032    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p75zz" event={"ID":"1092321a-6edf-43d5-bb31-5efb0b80990e","Type":"ContainerStarted","Data":"bdad564b5214dd0e075ebdfbad2026feb1e817d73634f68fc430fe432ec43496"}
Nov 02 01:31:58.206918 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:58.206895    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:31:58.207334 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:58.207151    4515 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 01:31:58.722423 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:58.722393    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:31:59.008712 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:59.008631    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p75zz" event={"ID":"1092321a-6edf-43d5-bb31-5efb0b80990e","Type":"ContainerStarted","Data":"1371d01531dd309fbc4f1e918e3d5b036b2538565f118333f27a1ba22458ac31"}
Nov 02 01:31:59.008712 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:59.008697    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p75zz" event={"ID":"1092321a-6edf-43d5-bb31-5efb0b80990e","Type":"ContainerStarted","Data":"81ccfa5c927eac57b61a6c8cbdcc20ff913371d0a45af608d4a0c3b96313cb09"}
Nov 02 01:31:59.762355 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:59.762313    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:59.762838 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:59.762359    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:59.764655 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:59.764628    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnkn6\" (UID: \"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4\") " pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:31:59.764775 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:59.764628    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c429f46-34a2-48a4-8fe5-8204fc5ff5cb-metrics-tls\") pod \"dns-default-dcc98\" (UID: \"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb\") " pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:59.810622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:59.810596    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dcc98"
Nov 02 01:31:59.810723 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:31:59.810653    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cnkn6"
Nov 02 01:32:00.289336 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:00.289292    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dcc98"]
Nov 02 01:32:00.293000 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:00.292974    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cnkn6"]
Nov 02 01:32:00.293593 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:32:00.293571    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c429f46_34a2_48a4_8fe5_8204fc5ff5cb.slice/crio-9c7140a9a336803f767b15c483f41fa83124365fdf881b22f9be4a28ab2c7864 WatchSource:0}: Error finding container 9c7140a9a336803f767b15c483f41fa83124365fdf881b22f9be4a28ab2c7864: Status 404 returned error can't find the container with id 9c7140a9a336803f767b15c483f41fa83124365fdf881b22f9be4a28ab2c7864
Nov 02 01:32:00.331969 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:32:00.331943    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b0cd0b8_7f6c_41d3_9237_33ad1ce6dbb4.slice/crio-ed7b69796b2e5431fbbbbddb9221888d44dec2396ec7de29d794bfc7dd4e63a9 WatchSource:0}: Error finding container ed7b69796b2e5431fbbbbddb9221888d44dec2396ec7de29d794bfc7dd4e63a9: Status 404 returned error can't find the container with id ed7b69796b2e5431fbbbbddb9221888d44dec2396ec7de29d794bfc7dd4e63a9
Nov 02 01:32:01.016383 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:01.016350    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cnkn6" event={"ID":"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4","Type":"ContainerStarted","Data":"be72220c6f31fcf30398fd515701e7aacbeef53f6f4e0aa4d250f118f8b19fef"}
Nov 02 01:32:01.016783 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:01.016416    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cnkn6" event={"ID":"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4","Type":"ContainerStarted","Data":"ed7b69796b2e5431fbbbbddb9221888d44dec2396ec7de29d794bfc7dd4e63a9"}
Nov 02 01:32:01.017410 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:01.017387    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dcc98" event={"ID":"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb","Type":"ContainerStarted","Data":"9c7140a9a336803f767b15c483f41fa83124365fdf881b22f9be4a28ab2c7864"}
Nov 02 01:32:01.465015 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:01.464982    4515 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-75.us-west-1.compute.internal" event="NodeReady"
Nov 02 01:32:02.021894 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:02.021859    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cnkn6" event={"ID":"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4","Type":"ContainerStarted","Data":"c3a32cf693732679795aa65da7c06df822ed6869f18ba79b94845761d3f959ad"}
Nov 02 01:32:04.031318 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.031281    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dcc98" event={"ID":"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb","Type":"ContainerStarted","Data":"2228fd0f4ccf9a24de6158d9267036f6b7f0829c3fc07c9db6d0e12d2895696c"}
Nov 02 01:32:04.031706 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.031343    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dcc98" event={"ID":"8c429f46-34a2-48a4-8fe5-8204fc5ff5cb","Type":"ContainerStarted","Data":"be1c03784359356b304ef121b9ed4749dacadb27a8310855aefb53cc3134797b"}
Nov 02 01:32:04.031706 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.031392    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dcc98"
Nov 02 01:32:04.720385 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.720356    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-597558bbdf-ll9jr"]
Nov 02 01:32:04.721998 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.721983    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.797591 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.797565    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91b0b763-50ff-4882-9f42-d679dd7c8aeb-registry-certificates\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.797760 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.797602    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91b0b763-50ff-4882-9f42-d679dd7c8aeb-image-registry-private-configuration\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.797760 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.797622    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-registry-tls\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.797760 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.797662    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91b0b763-50ff-4882-9f42-d679dd7c8aeb-ca-trust-extracted\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.797760 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.797695    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91b0b763-50ff-4882-9f42-d679dd7c8aeb-trusted-ca\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.797896 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.797803    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91b0b763-50ff-4882-9f42-d679dd7c8aeb-installation-pull-secrets\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.797896 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.797832    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24fpr\" (UniqueName: \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-kube-api-access-24fpr\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.797896 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.797855    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-bound-sa-token\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.873909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.873780    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 02 01:32:04.876981 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.876948    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 02 01:32:04.877165 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.877061    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lljjm\""
Nov 02 01:32:04.877165 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.877075    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 02 01:32:04.877743 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.877723    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 02 01:32:04.878107 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.878087    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-597558bbdf-ll9jr"]
Nov 02 01:32:04.898910 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.898886    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91b0b763-50ff-4882-9f42-d679dd7c8aeb-registry-certificates\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.899004 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.898915    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91b0b763-50ff-4882-9f42-d679dd7c8aeb-image-registry-private-configuration\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.899004 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.898932    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-registry-tls\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.899004 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.898947    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91b0b763-50ff-4882-9f42-d679dd7c8aeb-ca-trust-extracted\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.899004 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.898968    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91b0b763-50ff-4882-9f42-d679dd7c8aeb-trusted-ca\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.899220 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.899005    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91b0b763-50ff-4882-9f42-d679dd7c8aeb-installation-pull-secrets\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.899220 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.899037    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24fpr\" (UniqueName: \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-kube-api-access-24fpr\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.899220 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.899065    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-bound-sa-token\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.899516 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.899489    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91b0b763-50ff-4882-9f42-d679dd7c8aeb-ca-trust-extracted\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.899711 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.899688    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91b0b763-50ff-4882-9f42-d679dd7c8aeb-registry-certificates\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.900170 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.900147    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91b0b763-50ff-4882-9f42-d679dd7c8aeb-trusted-ca\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.901534 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.901515    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91b0b763-50ff-4882-9f42-d679dd7c8aeb-image-registry-private-configuration\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.901665 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.901643    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91b0b763-50ff-4882-9f42-d679dd7c8aeb-installation-pull-secrets\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.901751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.901727    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-registry-tls\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.961178 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.961107    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-bound-sa-token\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:04.961322 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:04.961306    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24fpr\" (UniqueName: \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-kube-api-access-24fpr\") pod \"image-registry-597558bbdf-ll9jr\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") " pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:05.031397 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.031376    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:05.036840 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.036815    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cnkn6" event={"ID":"3b0cd0b8-7f6c-41d3-9237-33ad1ce6dbb4","Type":"ContainerStarted","Data":"f36815e2eb625a8c6bd6fc070de66e833aba43ed7f66a609110e266a05805b7d"}
Nov 02 01:32:05.094109 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.094083    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6468bb74bc-89nlm"]
Nov 02 01:32:05.096086 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.096072    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.102407 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.102380    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-bound-sa-token\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.102493 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.102446    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-registry-tls\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.102674 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.102648    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-image-registry-private-configuration\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.102712 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.102692    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-installation-pull-secrets\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.102747 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.102734    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-ca-trust-extracted\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.102912 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.102881    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-registry-certificates\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.103277 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.103256    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-trusted-ca\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.103324 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.103307    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6h97\" (UniqueName: \"kubernetes.io/projected/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-kube-api-access-v6h97\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.158513 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.158451    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6468bb74bc-89nlm"]
Nov 02 01:32:05.204234 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.204204    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6h97\" (UniqueName: \"kubernetes.io/projected/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-kube-api-access-v6h97\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.204371 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.204242    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-bound-sa-token\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.204371 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.204267    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-registry-tls\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.204371 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.204296    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-image-registry-private-configuration\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.204371 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.204320    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-installation-pull-secrets\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.204371 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.204344    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-ca-trust-extracted\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.204617 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.204382    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-registry-certificates\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.204617 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.204416    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-trusted-ca\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.204923 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.204894    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-ca-trust-extracted\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.205413 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.205393    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-registry-certificates\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.205504 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.205475    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-trusted-ca\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.206781 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.206764    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-registry-tls\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.206860 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.206778    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-image-registry-private-configuration\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.206860 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.206851    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-installation-pull-secrets\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.264781 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.264757    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-bound-sa-token\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.265005 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.264988    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6h97\" (UniqueName: \"kubernetes.io/projected/e36ac033-e80f-4a78-9d75-8f60ccad2dcf-kube-api-access-v6h97\") pod \"image-registry-6468bb74bc-89nlm\" (UID: \"e36ac033-e80f-4a78-9d75-8f60ccad2dcf\") " pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.406848 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.406816    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:05.461723 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.459342    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-597558bbdf-ll9jr"]
Nov 02 01:32:05.461723 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:32:05.460929    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91b0b763_50ff_4882_9f42_d679dd7c8aeb.slice/crio-d3abc33edfc03fd763662281d4aeb234effbf243845ff08e326f10d046b5b8f1 WatchSource:0}: Error finding container d3abc33edfc03fd763662281d4aeb234effbf243845ff08e326f10d046b5b8f1: Status 404 returned error can't find the container with id d3abc33edfc03fd763662281d4aeb234effbf243845ff08e326f10d046b5b8f1
Nov 02 01:32:05.776807 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:05.776778    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6468bb74bc-89nlm"]
Nov 02 01:32:05.780533 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:32:05.780507    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode36ac033_e80f_4a78_9d75_8f60ccad2dcf.slice/crio-0e9503b25a9a172939cbe06789650117190315bcd81947bda586587c164f866b WatchSource:0}: Error finding container 0e9503b25a9a172939cbe06789650117190315bcd81947bda586587c164f866b: Status 404 returned error can't find the container with id 0e9503b25a9a172939cbe06789650117190315bcd81947bda586587c164f866b
Nov 02 01:32:06.012570 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:06.012541    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hbcss"
Nov 02 01:32:06.042244 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:06.042153    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-597558bbdf-ll9jr" event={"ID":"91b0b763-50ff-4882-9f42-d679dd7c8aeb","Type":"ContainerStarted","Data":"568aa2be4f3e3d0d52c9864735894c65a90d65c44d72e0779f0830e8ed45a56f"}
Nov 02 01:32:06.042244 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:06.042217    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-597558bbdf-ll9jr" event={"ID":"91b0b763-50ff-4882-9f42-d679dd7c8aeb","Type":"ContainerStarted","Data":"d3abc33edfc03fd763662281d4aeb234effbf243845ff08e326f10d046b5b8f1"}
Nov 02 01:32:06.042756 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:06.042263    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:06.043771 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:06.043739    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6468bb74bc-89nlm" event={"ID":"e36ac033-e80f-4a78-9d75-8f60ccad2dcf","Type":"ContainerStarted","Data":"46377eeff52615e4aee687bbfd54b58729b1cf12f27f8d6d3bbb72ef6c471526"}
Nov 02 01:32:06.043876 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:06.043788    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6468bb74bc-89nlm" event={"ID":"e36ac033-e80f-4a78-9d75-8f60ccad2dcf","Type":"ContainerStarted","Data":"0e9503b25a9a172939cbe06789650117190315bcd81947bda586587c164f866b"}
Nov 02 01:32:06.162446 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:06.162381    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-597558bbdf-ll9jr" podStartSLOduration=2.16236508 podStartE2EDuration="2.16236508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:32:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:32:06.161422148 +0000 UTC m=+15.373791956" watchObservedRunningTime="2026-11-02 01:32:06.16236508 +0000 UTC m=+15.374734885"
Nov 02 01:32:07.046838 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:07.046807    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:07.293396 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:07.293369    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gs99s"
Nov 02 01:32:07.569790 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:07.569740    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6468bb74bc-89nlm" podStartSLOduration=2.5697261940000002 podStartE2EDuration="2.569726194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:32:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:32:06.283222107 +0000 UTC m=+15.495591916" watchObservedRunningTime="2026-11-02 01:32:07.569726194 +0000 UTC m=+16.782095996"
Nov 02 01:32:09.959231 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:32:09.959185    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:32:14.039383 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:14.039351    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dcc98"
Nov 02 01:32:20.357724 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.357687    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-77nlh"]
Nov 02 01:32:20.408943 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.408910    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.500246 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.500206    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/553be019-6080-45d6-bfba-348f682814c1-metrics-client-ca\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.500411 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.500252    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/553be019-6080-45d6-bfba-348f682814c1-root\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.500411 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.500276    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/553be019-6080-45d6-bfba-348f682814c1-node-exporter-wtmp\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.500411 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.500380    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/553be019-6080-45d6-bfba-348f682814c1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.500535 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.500417    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/553be019-6080-45d6-bfba-348f682814c1-node-exporter-tls\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.500535 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.500439    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/553be019-6080-45d6-bfba-348f682814c1-node-exporter-textfile\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.500535 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.500519    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/553be019-6080-45d6-bfba-348f682814c1-sys\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.500642 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.500547    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2f7g\" (UniqueName: \"kubernetes.io/projected/553be019-6080-45d6-bfba-348f682814c1-kube-api-access-c2f7g\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.500642 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.500572    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/553be019-6080-45d6-bfba-348f682814c1-node-exporter-accelerators-collector-config\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.573588 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.573557    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 02 01:32:20.573588 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.573575    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 02 01:32:20.573588 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.573580    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6vdkx\""
Nov 02 01:32:20.573874 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.573596    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 02 01:32:20.573874 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.573763    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 02 01:32:20.574077 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.574063    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 02 01:32:20.574198 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.574187    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 02 01:32:20.600904 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.600876    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/553be019-6080-45d6-bfba-348f682814c1-sys\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.601035 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.600909    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2f7g\" (UniqueName: \"kubernetes.io/projected/553be019-6080-45d6-bfba-348f682814c1-kube-api-access-c2f7g\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.601035 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.600938    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/553be019-6080-45d6-bfba-348f682814c1-node-exporter-accelerators-collector-config\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.601035 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.600980    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/553be019-6080-45d6-bfba-348f682814c1-sys\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.601035 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.600992    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/553be019-6080-45d6-bfba-348f682814c1-metrics-client-ca\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.601035 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.601031    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/553be019-6080-45d6-bfba-348f682814c1-root\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.601281 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.601056    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/553be019-6080-45d6-bfba-348f682814c1-node-exporter-wtmp\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.601281 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.601090    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/553be019-6080-45d6-bfba-348f682814c1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.601281 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.601135    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/553be019-6080-45d6-bfba-348f682814c1-node-exporter-tls\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.601281 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.601162    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/553be019-6080-45d6-bfba-348f682814c1-node-exporter-textfile\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.601281 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.601167    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/553be019-6080-45d6-bfba-348f682814c1-root\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.601281 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.601273    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/553be019-6080-45d6-bfba-348f682814c1-node-exporter-wtmp\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.601572 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.601553    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/553be019-6080-45d6-bfba-348f682814c1-node-exporter-textfile\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.601687 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.601665    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/553be019-6080-45d6-bfba-348f682814c1-metrics-client-ca\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.601788 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.601717    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/553be019-6080-45d6-bfba-348f682814c1-node-exporter-accelerators-collector-config\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.603628 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.603607    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/553be019-6080-45d6-bfba-348f682814c1-node-exporter-tls\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.603678 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.603607    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/553be019-6080-45d6-bfba-348f682814c1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.661301 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.661234    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2f7g\" (UniqueName: \"kubernetes.io/projected/553be019-6080-45d6-bfba-348f682814c1-kube-api-access-c2f7g\") pod \"node-exporter-77nlh\" (UID: \"553be019-6080-45d6-bfba-348f682814c1\") " pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.759989 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:20.759958    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-77nlh"
Nov 02 01:32:20.790166 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:32:20.790133    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod553be019_6080_45d6_bfba_348f682814c1.slice/crio-bbf40d872bd8b627472953931843233df515d642be3b2fe0c4d24e6fd2a55f94 WatchSource:0}: Error finding container bbf40d872bd8b627472953931843233df515d642be3b2fe0c4d24e6fd2a55f94: Status 404 returned error can't find the container with id bbf40d872bd8b627472953931843233df515d642be3b2fe0c4d24e6fd2a55f94
Nov 02 01:32:21.089604 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:21.089563    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-77nlh" event={"ID":"553be019-6080-45d6-bfba-348f682814c1","Type":"ContainerStarted","Data":"bbf40d872bd8b627472953931843233df515d642be3b2fe0c4d24e6fd2a55f94"}
Nov 02 01:32:23.097336 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:23.097296    4515 generic.go:356] "Generic (PLEG): container finished" podID="553be019-6080-45d6-bfba-348f682814c1" containerID="094ddd8b851672be0d768b2e9c1203726bacecb760be5e3d250e422b69fbbea6" exitCode=0
Nov 02 01:32:23.097824 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:23.097366    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-77nlh" event={"ID":"553be019-6080-45d6-bfba-348f682814c1","Type":"ContainerDied","Data":"094ddd8b851672be0d768b2e9c1203726bacecb760be5e3d250e422b69fbbea6"}
Nov 02 01:32:24.102750 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:24.102707    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-77nlh" event={"ID":"553be019-6080-45d6-bfba-348f682814c1","Type":"ContainerStarted","Data":"0b4babd05180801ccf569d91c2f78b916acb5721a350681e356544430993dbfb"}
Nov 02 01:32:24.103163 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:24.102771    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-77nlh" event={"ID":"553be019-6080-45d6-bfba-348f682814c1","Type":"ContainerStarted","Data":"0061f65f932d04516ab7273a22d30a0d21aae3e043ead66f7581308dae98981e"}
Nov 02 01:32:24.271537 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:24.271482    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-77nlh" podStartSLOduration=2.315401207 podStartE2EDuration="4.27146551s" totalImagesPullingTime="1.876475618s" totalInitContainerRuntime="79.588685ms" isStatefulPod=true podCreationTimestamp="2026-11-02 01:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:32:24.270653287 +0000 UTC m=+33.483023094" watchObservedRunningTime="2026-11-02 01:32:24.27146551 +0000 UTC m=+33.483835315"
Nov 02 01:32:27.050594 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:27.050567    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:28.054571 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:28.054543    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6468bb74bc-89nlm"
Nov 02 01:32:28.276133 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:28.276096    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-597558bbdf-ll9jr"]
Nov 02 01:32:28.725714 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:28.725684    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nth82"
Nov 02 01:32:35.371145 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.371048    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bc6fdd69d-n4575"]
Nov 02 01:32:35.373014 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.372999    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.402071 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.402038    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81ad565a-ba22-4ef9-8bba-563612721294-console-serving-cert\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.402217 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.402087    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-console-config\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.402217 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.402200    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81ad565a-ba22-4ef9-8bba-563612721294-console-oauth-config\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.402304 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.402224    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-service-ca\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.402304 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.402275    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81ad565a-ba22-4ef9-8bba-563612721294-tmp\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.402410 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.402347    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6shgk\" (UniqueName: \"kubernetes.io/projected/81ad565a-ba22-4ef9-8bba-563612721294-kube-api-access-6shgk\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.402410 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.402382    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-oauth-serving-cert\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.437376 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.437347    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 02 01:32:35.437540 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.437385    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 02 01:32:35.437540 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.437421    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 02 01:32:35.487188 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.487154    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 02 01:32:35.487334 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.487250    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 02 01:32:35.487457 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.487443    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 02 01:32:35.487526 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.487476    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9fkx5\""
Nov 02 01:32:35.487526 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.487495    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 02 01:32:35.488447 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.488427    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bc6fdd69d-n4575"]
Nov 02 01:32:35.502740 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.502719    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-console-config\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.502810 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.502753    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81ad565a-ba22-4ef9-8bba-563612721294-console-oauth-config\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.502810 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.502768    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-service-ca\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.502810 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.502788    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81ad565a-ba22-4ef9-8bba-563612721294-tmp\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.502914 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.502821    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6shgk\" (UniqueName: \"kubernetes.io/projected/81ad565a-ba22-4ef9-8bba-563612721294-kube-api-access-6shgk\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.502914 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.502847    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-oauth-serving-cert\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.502914 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.502882    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81ad565a-ba22-4ef9-8bba-563612721294-console-serving-cert\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.503279 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.503248    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81ad565a-ba22-4ef9-8bba-563612721294-tmp\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.503485 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.503461    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-console-config\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.503627 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.503605    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-service-ca\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.503754 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.503733    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-oauth-serving-cert\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.505356 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.505338    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81ad565a-ba22-4ef9-8bba-563612721294-console-oauth-config\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.505409 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.505389    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81ad565a-ba22-4ef9-8bba-563612721294-console-serving-cert\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.563771 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.563746    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6shgk\" (UniqueName: \"kubernetes.io/projected/81ad565a-ba22-4ef9-8bba-563612721294-kube-api-access-6shgk\") pod \"console-6bc6fdd69d-n4575\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") " pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:35.685045 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:35.685017    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:36.053782 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:36.053706    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bc6fdd69d-n4575"]
Nov 02 01:32:36.056807 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:32:36.056782    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81ad565a_ba22_4ef9_8bba_563612721294.slice/crio-7ac00ab0df3857c41bec43df76af442db00db21efe7b13740a6abebebfa51705 WatchSource:0}: Error finding container 7ac00ab0df3857c41bec43df76af442db00db21efe7b13740a6abebebfa51705: Status 404 returned error can't find the container with id 7ac00ab0df3857c41bec43df76af442db00db21efe7b13740a6abebebfa51705
Nov 02 01:32:36.147141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:36.147097    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc6fdd69d-n4575" event={"ID":"81ad565a-ba22-4ef9-8bba-563612721294","Type":"ContainerStarted","Data":"7ac00ab0df3857c41bec43df76af442db00db21efe7b13740a6abebebfa51705"}
Nov 02 01:32:39.958319 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:32:39.958279    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:32:40.162362 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:40.162331    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc6fdd69d-n4575" event={"ID":"81ad565a-ba22-4ef9-8bba-563612721294","Type":"ContainerStarted","Data":"0d02014d6765e992f2485ea5f3efc957e50dcffbf99c65b7da29b62fcec60bd2"}
Nov 02 01:32:40.281964 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:40.281863    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6bc6fdd69d-n4575" podStartSLOduration=1.826836311 podStartE2EDuration="5.281848882s" totalImagesPullingTime="3.455012571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:32:40.281135685 +0000 UTC m=+49.493505491" watchObservedRunningTime="2026-11-02 01:32:40.281848882 +0000 UTC m=+49.494218759"
Nov 02 01:32:45.685847 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:45.685811    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:45.686334 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:45.685892    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:45.690807 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:45.690780    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:46.184887 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:46.184865    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:32:47.097280 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.097250    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bdbdbfb7b-bzj2l"]
Nov 02 01:32:47.100416 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.100400    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.186445 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.186414    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-trusted-ca-bundle\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.186606 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.186450    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-oauth-config\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.186606 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.186468    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55lq6\" (UniqueName: \"kubernetes.io/projected/8617cb05-8535-4ada-bee4-bbc8ec853a0f-kube-api-access-55lq6\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.186606 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.186484    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8617cb05-8535-4ada-bee4-bbc8ec853a0f-tmp\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.186606 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.186540    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-config\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.186746 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.186664    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-oauth-serving-cert\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.186746 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.186727    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-serving-cert\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.186812 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.186751    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-service-ca\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.287712 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.287677    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55lq6\" (UniqueName: \"kubernetes.io/projected/8617cb05-8535-4ada-bee4-bbc8ec853a0f-kube-api-access-55lq6\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.287712 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.287713    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8617cb05-8535-4ada-bee4-bbc8ec853a0f-tmp\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.287975 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.287743    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-config\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.287975 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.287828    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-oauth-serving-cert\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.287975 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.287915    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-serving-cert\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.287975 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.287965    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-service-ca\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.288237 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.288092    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-trusted-ca-bundle\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.288237 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.288176    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-oauth-config\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.288237 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.288212    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8617cb05-8535-4ada-bee4-bbc8ec853a0f-tmp\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.288589 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.288568    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-config\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.288720 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.288695    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-oauth-serving-cert\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.288833 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.288818    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-service-ca\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.290660 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.290636    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-oauth-config\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.290764 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.290665    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-serving-cert\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.301949 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.301927    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Nov 02 01:32:47.302878 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.302854    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bdbdbfb7b-bzj2l"]
Nov 02 01:32:47.309851 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.309818    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-trusted-ca-bundle\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.350643 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.350584    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55lq6\" (UniqueName: \"kubernetes.io/projected/8617cb05-8535-4ada-bee4-bbc8ec853a0f-kube-api-access-55lq6\") pod \"console-6bdbdbfb7b-bzj2l\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") " pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.411365 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.411330    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:47.783215 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:47.783186    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bdbdbfb7b-bzj2l"]
Nov 02 01:32:47.815243 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:32:47.815219    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8617cb05_8535_4ada_bee4_bbc8ec853a0f.slice/crio-3dd33789e973be85927d268ed7db99fc89ddbbe95504bebee398cfce44e7aca6 WatchSource:0}: Error finding container 3dd33789e973be85927d268ed7db99fc89ddbbe95504bebee398cfce44e7aca6: Status 404 returned error can't find the container with id 3dd33789e973be85927d268ed7db99fc89ddbbe95504bebee398cfce44e7aca6
Nov 02 01:32:48.191761 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:48.191725    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bdbdbfb7b-bzj2l" event={"ID":"8617cb05-8535-4ada-bee4-bbc8ec853a0f","Type":"ContainerStarted","Data":"1196ec7d8e30622b36185b394943703e1adfa91181c79b4f0271002df31d993c"}
Nov 02 01:32:48.192161 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:48.191816    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bdbdbfb7b-bzj2l" event={"ID":"8617cb05-8535-4ada-bee4-bbc8ec853a0f","Type":"ContainerStarted","Data":"3dd33789e973be85927d268ed7db99fc89ddbbe95504bebee398cfce44e7aca6"}
Nov 02 01:32:48.313479 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:48.313420    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6bdbdbfb7b-bzj2l" podStartSLOduration=1.3134052440000001 podStartE2EDuration="1.313405244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:32:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:32:48.312345841 +0000 UTC m=+57.524715650" watchObservedRunningTime="2026-11-02 01:32:48.313405244 +0000 UTC m=+57.525775133"
Nov 02 01:32:53.300469 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.300412    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-597558bbdf-ll9jr" podUID="91b0b763-50ff-4882-9f42-d679dd7c8aeb" containerName="registry" containerID="cri-o://568aa2be4f3e3d0d52c9864735894c65a90d65c44d72e0779f0830e8ed45a56f" gracePeriod=30
Nov 02 01:32:53.549976 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.549955    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:53.639508 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.639439    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-registry-tls\" (UniqueName: \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-registry-tls\") pod \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") "
Nov 02 01:32:53.639508 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.639493    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91b0b763-50ff-4882-9f42-d679dd7c8aeb-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91b0b763-50ff-4882-9f42-d679dd7c8aeb-installation-pull-secrets\") pod \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") "
Nov 02 01:32:53.639681 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.639525    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91b0b763-50ff-4882-9f42-d679dd7c8aeb-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91b0b763-50ff-4882-9f42-d679dd7c8aeb-ca-trust-extracted\") pod \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") "
Nov 02 01:32:53.639681 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.639561    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91b0b763-50ff-4882-9f42-d679dd7c8aeb-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91b0b763-50ff-4882-9f42-d679dd7c8aeb-trusted-ca\") pod \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") "
Nov 02 01:32:53.639681 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.639589    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91b0b763-50ff-4882-9f42-d679dd7c8aeb-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91b0b763-50ff-4882-9f42-d679dd7c8aeb-image-registry-private-configuration\") pod \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") "
Nov 02 01:32:53.639681 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.639627    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-kube-api-access-24fpr\" (UniqueName: \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-kube-api-access-24fpr\") pod \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") "
Nov 02 01:32:53.639681 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.639676    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91b0b763-50ff-4882-9f42-d679dd7c8aeb-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91b0b763-50ff-4882-9f42-d679dd7c8aeb-registry-certificates\") pod \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") "
Nov 02 01:32:53.639912 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.639702    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-bound-sa-token\") pod \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\" (UID: \"91b0b763-50ff-4882-9f42-d679dd7c8aeb\") "
Nov 02 01:32:53.640453 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.640037    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91b0b763-50ff-4882-9f42-d679dd7c8aeb-trusted-ca" pod "91b0b763-50ff-4882-9f42-d679dd7c8aeb" (UID: "91b0b763-50ff-4882-9f42-d679dd7c8aeb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:32:53.640453 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.640391    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91b0b763-50ff-4882-9f42-d679dd7c8aeb-registry-certificates" pod "91b0b763-50ff-4882-9f42-d679dd7c8aeb" (UID: "91b0b763-50ff-4882-9f42-d679dd7c8aeb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:32:53.642301 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.642252    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91b0b763-50ff-4882-9f42-d679dd7c8aeb-installation-pull-secrets" pod "91b0b763-50ff-4882-9f42-d679dd7c8aeb" (UID: "91b0b763-50ff-4882-9f42-d679dd7c8aeb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 01:32:53.642301 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.642271    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-registry-tls" pod "91b0b763-50ff-4882-9f42-d679dd7c8aeb" (UID: "91b0b763-50ff-4882-9f42-d679dd7c8aeb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 01:32:53.642301 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.642276    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91b0b763-50ff-4882-9f42-d679dd7c8aeb-image-registry-private-configuration" pod "91b0b763-50ff-4882-9f42-d679dd7c8aeb" (UID: "91b0b763-50ff-4882-9f42-d679dd7c8aeb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 01:32:53.642301 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.642290    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-kube-api-access-24fpr" pod "91b0b763-50ff-4882-9f42-d679dd7c8aeb" (UID: "91b0b763-50ff-4882-9f42-d679dd7c8aeb"). InnerVolumeSpecName "kube-api-access-24fpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 01:32:53.642492 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.642297    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-bound-sa-token" pod "91b0b763-50ff-4882-9f42-d679dd7c8aeb" (UID: "91b0b763-50ff-4882-9f42-d679dd7c8aeb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 01:32:53.648642 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.648613    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91b0b763-50ff-4882-9f42-d679dd7c8aeb-ca-trust-extracted" pod "91b0b763-50ff-4882-9f42-d679dd7c8aeb" (UID: "91b0b763-50ff-4882-9f42-d679dd7c8aeb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:32:53.740689 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.740649    4515 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91b0b763-50ff-4882-9f42-d679dd7c8aeb-trusted-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:32:53.740689 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.740685    4515 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91b0b763-50ff-4882-9f42-d679dd7c8aeb-image-registry-private-configuration\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:32:53.740887 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.740700    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24fpr\" (UniqueName: \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-kube-api-access-24fpr\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:32:53.740887 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.740714    4515 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91b0b763-50ff-4882-9f42-d679dd7c8aeb-registry-certificates\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:32:53.740887 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.740726    4515 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-bound-sa-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:32:53.740887 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.740736    4515 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91b0b763-50ff-4882-9f42-d679dd7c8aeb-registry-tls\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:32:53.740887 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.740748    4515 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91b0b763-50ff-4882-9f42-d679dd7c8aeb-installation-pull-secrets\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:32:53.740887 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:53.740760    4515 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91b0b763-50ff-4882-9f42-d679dd7c8aeb-ca-trust-extracted\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:32:54.213613 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:54.213580    4515 generic.go:356] "Generic (PLEG): container finished" podID="91b0b763-50ff-4882-9f42-d679dd7c8aeb" containerID="568aa2be4f3e3d0d52c9864735894c65a90d65c44d72e0779f0830e8ed45a56f" exitCode=0
Nov 02 01:32:54.213772 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:54.213625    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-597558bbdf-ll9jr"
Nov 02 01:32:54.213772 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:54.213646    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-597558bbdf-ll9jr" event={"ID":"91b0b763-50ff-4882-9f42-d679dd7c8aeb","Type":"ContainerDied","Data":"568aa2be4f3e3d0d52c9864735894c65a90d65c44d72e0779f0830e8ed45a56f"}
Nov 02 01:32:54.213772 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:54.213686    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-597558bbdf-ll9jr" event={"ID":"91b0b763-50ff-4882-9f42-d679dd7c8aeb","Type":"ContainerDied","Data":"d3abc33edfc03fd763662281d4aeb234effbf243845ff08e326f10d046b5b8f1"}
Nov 02 01:32:54.213772 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:54.213716    4515 scope.go:122] "RemoveContainer" containerID="568aa2be4f3e3d0d52c9864735894c65a90d65c44d72e0779f0830e8ed45a56f"
Nov 02 01:32:54.221794 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:54.221779    4515 scope.go:122] "RemoveContainer" containerID="568aa2be4f3e3d0d52c9864735894c65a90d65c44d72e0779f0830e8ed45a56f"
Nov 02 01:32:54.222048 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:32:54.222024    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"568aa2be4f3e3d0d52c9864735894c65a90d65c44d72e0779f0830e8ed45a56f\": container with ID starting with 568aa2be4f3e3d0d52c9864735894c65a90d65c44d72e0779f0830e8ed45a56f not found: ID does not exist" containerID="568aa2be4f3e3d0d52c9864735894c65a90d65c44d72e0779f0830e8ed45a56f"
Nov 02 01:32:54.222139 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:54.222061    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"568aa2be4f3e3d0d52c9864735894c65a90d65c44d72e0779f0830e8ed45a56f"} err="failed to get container status \"568aa2be4f3e3d0d52c9864735894c65a90d65c44d72e0779f0830e8ed45a56f\": rpc error: code = NotFound desc = could not find container \"568aa2be4f3e3d0d52c9864735894c65a90d65c44d72e0779f0830e8ed45a56f\": container with ID starting with 568aa2be4f3e3d0d52c9864735894c65a90d65c44d72e0779f0830e8ed45a56f not found: ID does not exist"
Nov 02 01:32:54.431787 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:54.429937    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-597558bbdf-ll9jr"]
Nov 02 01:32:54.478149 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:54.478068    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-597558bbdf-ll9jr"]
Nov 02 01:32:54.950404 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:54.950375    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91b0b763-50ff-4882-9f42-d679dd7c8aeb" path="/var/lib/kubelet/pods/91b0b763-50ff-4882-9f42-d679dd7c8aeb/volumes"
Nov 02 01:32:57.412000 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:57.411968    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:57.412429 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:57.412026    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:57.416661 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:57.416635    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:58.230267 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:58.230239    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:32:58.398658 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:32:58.398620    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bc6fdd69d-n4575"]
Nov 02 01:33:05.680340 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.680302    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-d969c6fb7-42snk"]
Nov 02 01:33:05.680734 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.680561    4515 cpu_manager.go:414] "containerMap: removing container" podUID="91b0b763-50ff-4882-9f42-d679dd7c8aeb" containerName="registry"
Nov 02 01:33:05.680734 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.680576    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91b0b763-50ff-4882-9f42-d679dd7c8aeb" containerName="registry"
Nov 02 01:33:05.680734 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.680630    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="91b0b763-50ff-4882-9f42-d679dd7c8aeb" containerName="registry"
Nov 02 01:33:05.683302 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.683286    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.743845 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.743816    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d969c6fb7-42snk"]
Nov 02 01:33:05.838103 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.838075    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7d90120-9ee6-4ddb-92e3-1059804ea534-tmp\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.838103 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.838106    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-oauth-config\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.838335 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.838153    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-config\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.838335 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.838266    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-serving-cert\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.838335 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.838293    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r6hf\" (UniqueName: \"kubernetes.io/projected/f7d90120-9ee6-4ddb-92e3-1059804ea534-kube-api-access-5r6hf\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.838482 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.838357    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-trusted-ca-bundle\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.838482 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.838438    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-oauth-serving-cert\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.838482 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.838467    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-service-ca\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.939541 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.939512    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-trusted-ca-bundle\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.939705 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.939561    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-oauth-serving-cert\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.939705 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.939590    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-service-ca\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.939705 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.939618    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7d90120-9ee6-4ddb-92e3-1059804ea534-tmp\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.939705 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.939645    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-oauth-config\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.939705 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.939697    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-config\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.939950 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.939729    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-serving-cert\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.939950 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.939753    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r6hf\" (UniqueName: \"kubernetes.io/projected/f7d90120-9ee6-4ddb-92e3-1059804ea534-kube-api-access-5r6hf\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.940407 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.940379    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-oauth-serving-cert\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.940518 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.940431    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-service-ca\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.940518 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.940487    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-trusted-ca-bundle\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.940629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.940572    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-config\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.940629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.940603    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7d90120-9ee6-4ddb-92e3-1059804ea534-tmp\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.942166 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.942139    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-oauth-config\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:05.942266 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:05.942250    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-serving-cert\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:06.000436 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:06.000412    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r6hf\" (UniqueName: \"kubernetes.io/projected/f7d90120-9ee6-4ddb-92e3-1059804ea534-kube-api-access-5r6hf\") pod \"console-d969c6fb7-42snk\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") " pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:06.293852 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:06.293768    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:06.666014 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:06.665924    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d969c6fb7-42snk"]
Nov 02 01:33:06.673788 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:33:06.673742    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7d90120_9ee6_4ddb_92e3_1059804ea534.slice/crio-1375cb6fe8e80a5323ff72f76e013f4550e5c1899cec878f84844cc95d3e54bf WatchSource:0}: Error finding container 1375cb6fe8e80a5323ff72f76e013f4550e5c1899cec878f84844cc95d3e54bf: Status 404 returned error can't find the container with id 1375cb6fe8e80a5323ff72f76e013f4550e5c1899cec878f84844cc95d3e54bf
Nov 02 01:33:07.259881 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:07.259845    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d969c6fb7-42snk" event={"ID":"f7d90120-9ee6-4ddb-92e3-1059804ea534","Type":"ContainerStarted","Data":"ef1f75195138bc9d3208546528a1ddf1a253aec7a8937967faeb064b6a05b61c"}
Nov 02 01:33:07.259881 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:07.259890    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d969c6fb7-42snk" event={"ID":"f7d90120-9ee6-4ddb-92e3-1059804ea534","Type":"ContainerStarted","Data":"1375cb6fe8e80a5323ff72f76e013f4550e5c1899cec878f84844cc95d3e54bf"}
Nov 02 01:33:09.956799 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:33:09.956765    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:33:16.294716 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:16.294681    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:16.295205 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:16.294936    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:16.299405 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:16.299384    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:16.417263 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:16.417218    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-d969c6fb7-42snk" podStartSLOduration=11.417206355 podStartE2EDuration="11.417206355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:33:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:33:07.378380559 +0000 UTC m=+76.590750367" watchObservedRunningTime="2026-11-02 01:33:16.417206355 +0000 UTC m=+85.629576155"
Nov 02 01:33:17.294566 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:17.294540    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:33:17.462402 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:17.462367    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bdbdbfb7b-bzj2l"]
Nov 02 01:33:19.925907 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:19.925871    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-hmn9t-cwkdz"]
Nov 02 01:33:19.926392 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:19.926070    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-hmn9t-cwkdz" podUID="fd401370-c917-4608-b6df-da7367f1ad21" containerName="insights-gathering" containerID="cri-o://3ec47fccde4f4b2d63c73fd56892eaa0ec8a13d013eba2ea242acb5716c0a19c" gracePeriod=30
Nov 02 01:33:20.167133 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.167097    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:33:20.300303 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.300270    4515 generic.go:356] "Generic (PLEG): container finished" podID="fd401370-c917-4608-b6df-da7367f1ad21" containerID="3ec47fccde4f4b2d63c73fd56892eaa0ec8a13d013eba2ea242acb5716c0a19c" exitCode=2
Nov 02 01:33:20.300459 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.300333    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-hmn9t-cwkdz" event={"ID":"fd401370-c917-4608-b6df-da7367f1ad21","Type":"ContainerDied","Data":"3ec47fccde4f4b2d63c73fd56892eaa0ec8a13d013eba2ea242acb5716c0a19c"}
Nov 02 01:33:20.300459 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.300340    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-hmn9t-cwkdz"
Nov 02 01:33:20.300459 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.300375    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-hmn9t-cwkdz" event={"ID":"fd401370-c917-4608-b6df-da7367f1ad21","Type":"ContainerDied","Data":"a2484711bd47b284c5c8facb3a5b7bd081e5b3315c2a3d92aa88acd422621b25"}
Nov 02 01:33:20.300459 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.300395    4515 scope.go:122] "RemoveContainer" containerID="3ec47fccde4f4b2d63c73fd56892eaa0ec8a13d013eba2ea242acb5716c0a19c"
Nov 02 01:33:20.307961 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.307941    4515 scope.go:122] "RemoveContainer" containerID="3ec47fccde4f4b2d63c73fd56892eaa0ec8a13d013eba2ea242acb5716c0a19c"
Nov 02 01:33:20.308226 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:33:20.308206    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ec47fccde4f4b2d63c73fd56892eaa0ec8a13d013eba2ea242acb5716c0a19c\": container with ID starting with 3ec47fccde4f4b2d63c73fd56892eaa0ec8a13d013eba2ea242acb5716c0a19c not found: ID does not exist" containerID="3ec47fccde4f4b2d63c73fd56892eaa0ec8a13d013eba2ea242acb5716c0a19c"
Nov 02 01:33:20.308325 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.308235    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec47fccde4f4b2d63c73fd56892eaa0ec8a13d013eba2ea242acb5716c0a19c"} err="failed to get container status \"3ec47fccde4f4b2d63c73fd56892eaa0ec8a13d013eba2ea242acb5716c0a19c\": rpc error: code = NotFound desc = could not find container \"3ec47fccde4f4b2d63c73fd56892eaa0ec8a13d013eba2ea242acb5716c0a19c\": container with ID starting with 3ec47fccde4f4b2d63c73fd56892eaa0ec8a13d013eba2ea242acb5716c0a19c not found: ID does not exist"
Nov 02 01:33:20.350716 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.350684    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-service-ca-bundle\") pod \"fd401370-c917-4608-b6df-da7367f1ad21\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") "
Nov 02 01:33:20.350839 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.350739    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd401370-c917-4608-b6df-da7367f1ad21-kube-api-access-c6pv9\" (UniqueName: \"kubernetes.io/projected/fd401370-c917-4608-b6df-da7367f1ad21-kube-api-access-c6pv9\") pod \"fd401370-c917-4608-b6df-da7367f1ad21\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") "
Nov 02 01:33:20.350839 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.350762    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-trusted-ca-bundle\") pod \"fd401370-c917-4608-b6df-da7367f1ad21\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") "
Nov 02 01:33:20.350928 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.350883    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd401370-c917-4608-b6df-da7367f1ad21-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fd401370-c917-4608-b6df-da7367f1ad21-archives-path\") pod \"fd401370-c917-4608-b6df-da7367f1ad21\" (UID: \"fd401370-c917-4608-b6df-da7367f1ad21\") "
Nov 02 01:33:20.351076 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.351061    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-trusted-ca-bundle" pod "fd401370-c917-4608-b6df-da7367f1ad21" (UID: "fd401370-c917-4608-b6df-da7367f1ad21"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:33:20.351160 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.351137    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-service-ca-bundle" pod "fd401370-c917-4608-b6df-da7367f1ad21" (UID: "fd401370-c917-4608-b6df-da7367f1ad21"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:33:20.351206 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.351167    4515 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-trusted-ca-bundle\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:20.351254 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.351193    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd401370-c917-4608-b6df-da7367f1ad21-archives-path" pod "fd401370-c917-4608-b6df-da7367f1ad21" (UID: "fd401370-c917-4608-b6df-da7367f1ad21"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:33:20.352979 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.352960    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd401370-c917-4608-b6df-da7367f1ad21-kube-api-access-c6pv9" pod "fd401370-c917-4608-b6df-da7367f1ad21" (UID: "fd401370-c917-4608-b6df-da7367f1ad21"). InnerVolumeSpecName "kube-api-access-c6pv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 01:33:20.451478 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.451449    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6pv9\" (UniqueName: \"kubernetes.io/projected/fd401370-c917-4608-b6df-da7367f1ad21-kube-api-access-c6pv9\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:20.451478 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.451477    4515 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fd401370-c917-4608-b6df-da7367f1ad21-archives-path\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:20.451633 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.451488    4515 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd401370-c917-4608-b6df-da7367f1ad21-service-ca-bundle\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:20.718276 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.718243    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-hmn9t-cwkdz"]
Nov 02 01:33:20.767887 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.767856    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-hmn9t-cwkdz"]
Nov 02 01:33:20.951773 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:20.951741    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd401370-c917-4608-b6df-da7367f1ad21" path="/var/lib/kubelet/pods/fd401370-c917-4608-b6df-da7367f1ad21/volumes"
Nov 02 01:33:23.417971 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.417934    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6bc6fdd69d-n4575" podUID="81ad565a-ba22-4ef9-8bba-563612721294" containerName="console" containerID="cri-o://0d02014d6765e992f2485ea5f3efc957e50dcffbf99c65b7da29b62fcec60bd2" gracePeriod=15
Nov 02 01:33:23.668823 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.668775    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:33:23.776276 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.776231    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81ad565a-ba22-4ef9-8bba-563612721294-kube-api-access-6shgk\" (UniqueName: \"kubernetes.io/projected/81ad565a-ba22-4ef9-8bba-563612721294-kube-api-access-6shgk\") pod \"81ad565a-ba22-4ef9-8bba-563612721294\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") "
Nov 02 01:33:23.776446 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.776299    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81ad565a-ba22-4ef9-8bba-563612721294-tmp\" (UniqueName: \"kubernetes.io/empty-dir/81ad565a-ba22-4ef9-8bba-563612721294-tmp\") pod \"81ad565a-ba22-4ef9-8bba-563612721294\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") "
Nov 02 01:33:23.776446 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.776321    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-service-ca\" (UniqueName: \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-service-ca\") pod \"81ad565a-ba22-4ef9-8bba-563612721294\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") "
Nov 02 01:33:23.776446 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.776346    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-oauth-serving-cert\") pod \"81ad565a-ba22-4ef9-8bba-563612721294\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") "
Nov 02 01:33:23.776446 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.776381    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/81ad565a-ba22-4ef9-8bba-563612721294-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81ad565a-ba22-4ef9-8bba-563612721294-console-oauth-config\") pod \"81ad565a-ba22-4ef9-8bba-563612721294\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") "
Nov 02 01:33:23.776446 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.776408    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-console-config\" (UniqueName: \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-console-config\") pod \"81ad565a-ba22-4ef9-8bba-563612721294\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") "
Nov 02 01:33:23.776689 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.776475    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/81ad565a-ba22-4ef9-8bba-563612721294-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81ad565a-ba22-4ef9-8bba-563612721294-console-serving-cert\") pod \"81ad565a-ba22-4ef9-8bba-563612721294\" (UID: \"81ad565a-ba22-4ef9-8bba-563612721294\") "
Nov 02 01:33:23.776758 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.776688    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81ad565a-ba22-4ef9-8bba-563612721294-tmp" pod "81ad565a-ba22-4ef9-8bba-563612721294" (UID: "81ad565a-ba22-4ef9-8bba-563612721294"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:33:23.776812 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.776794    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-service-ca" pod "81ad565a-ba22-4ef9-8bba-563612721294" (UID: "81ad565a-ba22-4ef9-8bba-563612721294"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:33:23.776867 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.776797    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-oauth-serving-cert" pod "81ad565a-ba22-4ef9-8bba-563612721294" (UID: "81ad565a-ba22-4ef9-8bba-563612721294"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:33:23.776919 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.776861    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-console-config" pod "81ad565a-ba22-4ef9-8bba-563612721294" (UID: "81ad565a-ba22-4ef9-8bba-563612721294"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:33:23.778620 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.778596    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81ad565a-ba22-4ef9-8bba-563612721294-console-serving-cert" pod "81ad565a-ba22-4ef9-8bba-563612721294" (UID: "81ad565a-ba22-4ef9-8bba-563612721294"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 01:33:23.778620 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.778609    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81ad565a-ba22-4ef9-8bba-563612721294-kube-api-access-6shgk" pod "81ad565a-ba22-4ef9-8bba-563612721294" (UID: "81ad565a-ba22-4ef9-8bba-563612721294"). InnerVolumeSpecName "kube-api-access-6shgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 01:33:23.778620 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.778612    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81ad565a-ba22-4ef9-8bba-563612721294-console-oauth-config" pod "81ad565a-ba22-4ef9-8bba-563612721294" (UID: "81ad565a-ba22-4ef9-8bba-563612721294"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 01:33:23.877441 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.877404    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6shgk\" (UniqueName: \"kubernetes.io/projected/81ad565a-ba22-4ef9-8bba-563612721294-kube-api-access-6shgk\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:23.877441 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.877435    4515 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81ad565a-ba22-4ef9-8bba-563612721294-tmp\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:23.877441 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.877446    4515 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-service-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:23.877667 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.877455    4515 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-oauth-serving-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:23.877667 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.877464    4515 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81ad565a-ba22-4ef9-8bba-563612721294-console-oauth-config\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:23.877667 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.877484    4515 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81ad565a-ba22-4ef9-8bba-563612721294-console-config\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:23.877667 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:23.877493    4515 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81ad565a-ba22-4ef9-8bba-563612721294-console-serving-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:24.315278 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:24.315247    4515 generic.go:356] "Generic (PLEG): container finished" podID="81ad565a-ba22-4ef9-8bba-563612721294" containerID="0d02014d6765e992f2485ea5f3efc957e50dcffbf99c65b7da29b62fcec60bd2" exitCode=0
Nov 02 01:33:24.315451 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:24.315300    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc6fdd69d-n4575"
Nov 02 01:33:24.315451 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:24.315308    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc6fdd69d-n4575" event={"ID":"81ad565a-ba22-4ef9-8bba-563612721294","Type":"ContainerDied","Data":"0d02014d6765e992f2485ea5f3efc957e50dcffbf99c65b7da29b62fcec60bd2"}
Nov 02 01:33:24.315451 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:24.315347    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc6fdd69d-n4575" event={"ID":"81ad565a-ba22-4ef9-8bba-563612721294","Type":"ContainerDied","Data":"7ac00ab0df3857c41bec43df76af442db00db21efe7b13740a6abebebfa51705"}
Nov 02 01:33:24.315451 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:24.315367    4515 scope.go:122] "RemoveContainer" containerID="0d02014d6765e992f2485ea5f3efc957e50dcffbf99c65b7da29b62fcec60bd2"
Nov 02 01:33:24.322795 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:24.322772    4515 scope.go:122] "RemoveContainer" containerID="0d02014d6765e992f2485ea5f3efc957e50dcffbf99c65b7da29b62fcec60bd2"
Nov 02 01:33:24.323209 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:33:24.323185    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d02014d6765e992f2485ea5f3efc957e50dcffbf99c65b7da29b62fcec60bd2\": container with ID starting with 0d02014d6765e992f2485ea5f3efc957e50dcffbf99c65b7da29b62fcec60bd2 not found: ID does not exist" containerID="0d02014d6765e992f2485ea5f3efc957e50dcffbf99c65b7da29b62fcec60bd2"
Nov 02 01:33:24.323287 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:24.323221    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d02014d6765e992f2485ea5f3efc957e50dcffbf99c65b7da29b62fcec60bd2"} err="failed to get container status \"0d02014d6765e992f2485ea5f3efc957e50dcffbf99c65b7da29b62fcec60bd2\": rpc error: code = NotFound desc = could not find container \"0d02014d6765e992f2485ea5f3efc957e50dcffbf99c65b7da29b62fcec60bd2\": container with ID starting with 0d02014d6765e992f2485ea5f3efc957e50dcffbf99c65b7da29b62fcec60bd2 not found: ID does not exist"
Nov 02 01:33:24.478632 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:24.478603    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bc6fdd69d-n4575"]
Nov 02 01:33:24.528221 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:24.528190    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bc6fdd69d-n4575"]
Nov 02 01:33:24.950256 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:24.950226    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81ad565a-ba22-4ef9-8bba-563612721294" path="/var/lib/kubelet/pods/81ad565a-ba22-4ef9-8bba-563612721294/volumes"
Nov 02 01:33:39.957645 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:33:39.957602    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:33:42.481185 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.481109    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6bdbdbfb7b-bzj2l" podUID="8617cb05-8535-4ada-bee4-bbc8ec853a0f" containerName="console" containerID="cri-o://1196ec7d8e30622b36185b394943703e1adfa91181c79b4f0271002df31d993c" gracePeriod=15
Nov 02 01:33:42.719711 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.719687    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:33:42.803644 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.803554    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-oauth-config\") pod \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") "
Nov 02 01:33:42.803644 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.803596    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-config\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-config\") pod \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") "
Nov 02 01:33:42.803644 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.803634    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-oauth-serving-cert\") pod \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") "
Nov 02 01:33:42.803918 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.803665    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8617cb05-8535-4ada-bee4-bbc8ec853a0f-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8617cb05-8535-4ada-bee4-bbc8ec853a0f-tmp\") pod \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") "
Nov 02 01:33:42.803918 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.803704    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-trusted-ca-bundle\") pod \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") "
Nov 02 01:33:42.803918 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.803735    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8617cb05-8535-4ada-bee4-bbc8ec853a0f-kube-api-access-55lq6\" (UniqueName: \"kubernetes.io/projected/8617cb05-8535-4ada-bee4-bbc8ec853a0f-kube-api-access-55lq6\") pod \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") "
Nov 02 01:33:42.803918 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.803779    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-serving-cert\") pod \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") "
Nov 02 01:33:42.803918 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.803811    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-service-ca\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-service-ca\") pod \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\" (UID: \"8617cb05-8535-4ada-bee4-bbc8ec853a0f\") "
Nov 02 01:33:42.804185 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.804074    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8617cb05-8535-4ada-bee4-bbc8ec853a0f-tmp" pod "8617cb05-8535-4ada-bee4-bbc8ec853a0f" (UID: "8617cb05-8535-4ada-bee4-bbc8ec853a0f"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:33:42.804242 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.804179    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-config" pod "8617cb05-8535-4ada-bee4-bbc8ec853a0f" (UID: "8617cb05-8535-4ada-bee4-bbc8ec853a0f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:33:42.804242 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.804214    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-trusted-ca-bundle" pod "8617cb05-8535-4ada-bee4-bbc8ec853a0f" (UID: "8617cb05-8535-4ada-bee4-bbc8ec853a0f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:33:42.804345 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.804258    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-oauth-serving-cert" pod "8617cb05-8535-4ada-bee4-bbc8ec853a0f" (UID: "8617cb05-8535-4ada-bee4-bbc8ec853a0f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:33:42.804345 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.804321    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-service-ca" pod "8617cb05-8535-4ada-bee4-bbc8ec853a0f" (UID: "8617cb05-8535-4ada-bee4-bbc8ec853a0f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:33:42.805805 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.805779    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-oauth-config" pod "8617cb05-8535-4ada-bee4-bbc8ec853a0f" (UID: "8617cb05-8535-4ada-bee4-bbc8ec853a0f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 01:33:42.805917 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.805890    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-serving-cert" pod "8617cb05-8535-4ada-bee4-bbc8ec853a0f" (UID: "8617cb05-8535-4ada-bee4-bbc8ec853a0f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 01:33:42.805969 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.805951    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8617cb05-8535-4ada-bee4-bbc8ec853a0f-kube-api-access-55lq6" pod "8617cb05-8535-4ada-bee4-bbc8ec853a0f" (UID: "8617cb05-8535-4ada-bee4-bbc8ec853a0f"). InnerVolumeSpecName "kube-api-access-55lq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 01:33:42.905247 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.905209    4515 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-oauth-config\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:42.905247 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.905240    4515 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-config\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:42.905247 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.905249    4515 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-oauth-serving-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:42.905636 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.905259    4515 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8617cb05-8535-4ada-bee4-bbc8ec853a0f-tmp\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:42.905636 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.905268    4515 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-trusted-ca-bundle\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:42.905636 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.905277    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55lq6\" (UniqueName: \"kubernetes.io/projected/8617cb05-8535-4ada-bee4-bbc8ec853a0f-kube-api-access-55lq6\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:42.905636 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.905286    4515 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8617cb05-8535-4ada-bee4-bbc8ec853a0f-console-serving-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:42.905636 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:42.905295    4515 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8617cb05-8535-4ada-bee4-bbc8ec853a0f-service-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:33:43.378298 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:43.378260    4515 generic.go:356] "Generic (PLEG): container finished" podID="8617cb05-8535-4ada-bee4-bbc8ec853a0f" containerID="1196ec7d8e30622b36185b394943703e1adfa91181c79b4f0271002df31d993c" exitCode=0
Nov 02 01:33:43.378466 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:43.378323    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bdbdbfb7b-bzj2l"
Nov 02 01:33:43.378466 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:43.378331    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bdbdbfb7b-bzj2l" event={"ID":"8617cb05-8535-4ada-bee4-bbc8ec853a0f","Type":"ContainerDied","Data":"1196ec7d8e30622b36185b394943703e1adfa91181c79b4f0271002df31d993c"}
Nov 02 01:33:43.378466 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:43.378370    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bdbdbfb7b-bzj2l" event={"ID":"8617cb05-8535-4ada-bee4-bbc8ec853a0f","Type":"ContainerDied","Data":"3dd33789e973be85927d268ed7db99fc89ddbbe95504bebee398cfce44e7aca6"}
Nov 02 01:33:43.378466 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:43.378388    4515 scope.go:122] "RemoveContainer" containerID="1196ec7d8e30622b36185b394943703e1adfa91181c79b4f0271002df31d993c"
Nov 02 01:33:43.385887 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:43.385872    4515 scope.go:122] "RemoveContainer" containerID="1196ec7d8e30622b36185b394943703e1adfa91181c79b4f0271002df31d993c"
Nov 02 01:33:43.386163 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:33:43.386143    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1196ec7d8e30622b36185b394943703e1adfa91181c79b4f0271002df31d993c\": container with ID starting with 1196ec7d8e30622b36185b394943703e1adfa91181c79b4f0271002df31d993c not found: ID does not exist" containerID="1196ec7d8e30622b36185b394943703e1adfa91181c79b4f0271002df31d993c"
Nov 02 01:33:43.386237 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:43.386171    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1196ec7d8e30622b36185b394943703e1adfa91181c79b4f0271002df31d993c"} err="failed to get container status \"1196ec7d8e30622b36185b394943703e1adfa91181c79b4f0271002df31d993c\": rpc error: code = NotFound desc = could not find container \"1196ec7d8e30622b36185b394943703e1adfa91181c79b4f0271002df31d993c\": container with ID starting with 1196ec7d8e30622b36185b394943703e1adfa91181c79b4f0271002df31d993c not found: ID does not exist"
Nov 02 01:33:43.540733 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:43.540704    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bdbdbfb7b-bzj2l"]
Nov 02 01:33:43.590522 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:43.590497    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bdbdbfb7b-bzj2l"]
Nov 02 01:33:44.950585 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:33:44.950554    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8617cb05-8535-4ada-bee4-bbc8ec853a0f" path="/var/lib/kubelet/pods/8617cb05-8535-4ada-bee4-bbc8ec853a0f/volumes"
Nov 02 01:34:04.759789 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.759716    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6fb9c77549-r257d"]
Nov 02 01:34:04.760200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.759941    4515 cpu_manager.go:414] "containerMap: removing container" podUID="8617cb05-8535-4ada-bee4-bbc8ec853a0f" containerName="console"
Nov 02 01:34:04.760200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.759952    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8617cb05-8535-4ada-bee4-bbc8ec853a0f" containerName="console"
Nov 02 01:34:04.760200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.759966    4515 cpu_manager.go:414] "containerMap: removing container" podUID="81ad565a-ba22-4ef9-8bba-563612721294" containerName="console"
Nov 02 01:34:04.760200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.759972    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81ad565a-ba22-4ef9-8bba-563612721294" containerName="console"
Nov 02 01:34:04.760200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.759978    4515 cpu_manager.go:414] "containerMap: removing container" podUID="fd401370-c917-4608-b6df-da7367f1ad21" containerName="insights-gathering"
Nov 02 01:34:04.760200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.759983    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd401370-c917-4608-b6df-da7367f1ad21" containerName="insights-gathering"
Nov 02 01:34:04.760200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.760028    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="8617cb05-8535-4ada-bee4-bbc8ec853a0f" containerName="console"
Nov 02 01:34:04.760200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.760035    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="81ad565a-ba22-4ef9-8bba-563612721294" containerName="console"
Nov 02 01:34:04.760200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.760040    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd401370-c917-4608-b6df-da7367f1ad21" containerName="insights-gathering"
Nov 02 01:34:04.762726 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.762710    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.848611 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.848568    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb1c2f26-6e49-49db-87df-11a760b2efbc-console-oauth-config\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.848611 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.848607    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb1c2f26-6e49-49db-87df-11a760b2efbc-trusted-ca-bundle\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.848816 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.848632    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb1c2f26-6e49-49db-87df-11a760b2efbc-oauth-serving-cert\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.848816 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.848758    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrpn9\" (UniqueName: \"kubernetes.io/projected/bb1c2f26-6e49-49db-87df-11a760b2efbc-kube-api-access-lrpn9\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.848900 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.848814    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1c2f26-6e49-49db-87df-11a760b2efbc-service-ca\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.848943 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.848920    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb1c2f26-6e49-49db-87df-11a760b2efbc-console-serving-cert\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.848982 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.848944    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb1c2f26-6e49-49db-87df-11a760b2efbc-tmp\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.848982 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.848965    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb1c2f26-6e49-49db-87df-11a760b2efbc-console-config\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.864761 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.864728    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fb9c77549-r257d"]
Nov 02 01:34:04.949269 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.949238    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrpn9\" (UniqueName: \"kubernetes.io/projected/bb1c2f26-6e49-49db-87df-11a760b2efbc-kube-api-access-lrpn9\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.949498 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.949279    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1c2f26-6e49-49db-87df-11a760b2efbc-service-ca\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.949498 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.949312    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb1c2f26-6e49-49db-87df-11a760b2efbc-console-serving-cert\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.949498 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.949326    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb1c2f26-6e49-49db-87df-11a760b2efbc-tmp\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.949498 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.949348    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb1c2f26-6e49-49db-87df-11a760b2efbc-console-config\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.949498 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.949380    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb1c2f26-6e49-49db-87df-11a760b2efbc-console-oauth-config\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.949498 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.949401    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb1c2f26-6e49-49db-87df-11a760b2efbc-trusted-ca-bundle\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.949498 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.949426    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb1c2f26-6e49-49db-87df-11a760b2efbc-oauth-serving-cert\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.950140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.950099    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1c2f26-6e49-49db-87df-11a760b2efbc-service-ca\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.950333 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.950222    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb1c2f26-6e49-49db-87df-11a760b2efbc-console-config\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.950461 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.950279    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb1c2f26-6e49-49db-87df-11a760b2efbc-oauth-serving-cert\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.950461 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.950428    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb1c2f26-6e49-49db-87df-11a760b2efbc-tmp\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.950559 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.950532    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb1c2f26-6e49-49db-87df-11a760b2efbc-trusted-ca-bundle\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.952637 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.952617    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb1c2f26-6e49-49db-87df-11a760b2efbc-console-serving-cert\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:04.952729 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:04.952710    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb1c2f26-6e49-49db-87df-11a760b2efbc-console-oauth-config\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:05.009733 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:05.009705    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrpn9\" (UniqueName: \"kubernetes.io/projected/bb1c2f26-6e49-49db-87df-11a760b2efbc-kube-api-access-lrpn9\") pod \"console-6fb9c77549-r257d\" (UID: \"bb1c2f26-6e49-49db-87df-11a760b2efbc\") " pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:05.073501 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:05.073424    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:05.443613 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:05.443580    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fb9c77549-r257d"]
Nov 02 01:34:05.447540 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:34:05.447517    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb1c2f26_6e49_49db_87df_11a760b2efbc.slice/crio-28a339318bb95cfd72d652f0e1e134155616ffcff4243fed4e363aed325026af WatchSource:0}: Error finding container 28a339318bb95cfd72d652f0e1e134155616ffcff4243fed4e363aed325026af: Status 404 returned error can't find the container with id 28a339318bb95cfd72d652f0e1e134155616ffcff4243fed4e363aed325026af
Nov 02 01:34:06.450053 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:06.450016    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fb9c77549-r257d" event={"ID":"bb1c2f26-6e49-49db-87df-11a760b2efbc","Type":"ContainerStarted","Data":"be9dd1fe331ee9dca8c6ba19fa12381c8f255142e07850a39a3c3f81efce3f19"}
Nov 02 01:34:06.450563 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:06.450065    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fb9c77549-r257d" event={"ID":"bb1c2f26-6e49-49db-87df-11a760b2efbc","Type":"ContainerStarted","Data":"28a339318bb95cfd72d652f0e1e134155616ffcff4243fed4e363aed325026af"}
Nov 02 01:34:06.621244 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:06.621188    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6fb9c77549-r257d" podStartSLOduration=2.621173605 podStartE2EDuration="2.621173605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:34:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:34:06.571440692 +0000 UTC m=+135.783810491" watchObservedRunningTime="2026-11-02 01:34:06.621173605 +0000 UTC m=+135.833543412"
Nov 02 01:34:09.957603 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:34:09.957567    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:34:15.074375 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:15.074340    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:15.074793 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:15.074395    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:15.079398 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:15.079375    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:15.484315 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:15.484281    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6fb9c77549-r257d"
Nov 02 01:34:15.660328 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:15.660299    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d969c6fb7-42snk"]
Nov 02 01:34:39.956635 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:34:39.956593    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:34:40.679075 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:40.679020    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-d969c6fb7-42snk" podUID="f7d90120-9ee6-4ddb-92e3-1059804ea534" containerName="console" containerID="cri-o://ef1f75195138bc9d3208546528a1ddf1a253aec7a8937967faeb064b6a05b61c" gracePeriod=15
Nov 02 01:34:40.917111 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:40.917087    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:34:41.003077 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.003047    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-oauth-serving-cert\") pod \"f7d90120-9ee6-4ddb-92e3-1059804ea534\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") "
Nov 02 01:34:41.003516 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.003084    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-trusted-ca-bundle\") pod \"f7d90120-9ee6-4ddb-92e3-1059804ea534\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") "
Nov 02 01:34:41.003516 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.003146    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-oauth-config\") pod \"f7d90120-9ee6-4ddb-92e3-1059804ea534\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") "
Nov 02 01:34:41.003516 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.003180    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7d90120-9ee6-4ddb-92e3-1059804ea534-kube-api-access-5r6hf\" (UniqueName: \"kubernetes.io/projected/f7d90120-9ee6-4ddb-92e3-1059804ea534-kube-api-access-5r6hf\") pod \"f7d90120-9ee6-4ddb-92e3-1059804ea534\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") "
Nov 02 01:34:41.003516 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.003206    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7d90120-9ee6-4ddb-92e3-1059804ea534-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7d90120-9ee6-4ddb-92e3-1059804ea534-tmp\") pod \"f7d90120-9ee6-4ddb-92e3-1059804ea534\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") "
Nov 02 01:34:41.003516 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.003240    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-serving-cert\") pod \"f7d90120-9ee6-4ddb-92e3-1059804ea534\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") "
Nov 02 01:34:41.003516 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.003274    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-config\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-config\") pod \"f7d90120-9ee6-4ddb-92e3-1059804ea534\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") "
Nov 02 01:34:41.003516 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.003299    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-service-ca\") pod \"f7d90120-9ee6-4ddb-92e3-1059804ea534\" (UID: \"f7d90120-9ee6-4ddb-92e3-1059804ea534\") "
Nov 02 01:34:41.003851 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.003513    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-oauth-serving-cert" pod "f7d90120-9ee6-4ddb-92e3-1059804ea534" (UID: "f7d90120-9ee6-4ddb-92e3-1059804ea534"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:34:41.003851 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.003519    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7d90120-9ee6-4ddb-92e3-1059804ea534-tmp" pod "f7d90120-9ee6-4ddb-92e3-1059804ea534" (UID: "f7d90120-9ee6-4ddb-92e3-1059804ea534"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:34:41.003851 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.003602    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-trusted-ca-bundle" pod "f7d90120-9ee6-4ddb-92e3-1059804ea534" (UID: "f7d90120-9ee6-4ddb-92e3-1059804ea534"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:34:41.003851 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.003810    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-config" pod "f7d90120-9ee6-4ddb-92e3-1059804ea534" (UID: "f7d90120-9ee6-4ddb-92e3-1059804ea534"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:34:41.004042 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.003861    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-service-ca" pod "f7d90120-9ee6-4ddb-92e3-1059804ea534" (UID: "f7d90120-9ee6-4ddb-92e3-1059804ea534"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:34:41.005348 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.005326    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-oauth-config" pod "f7d90120-9ee6-4ddb-92e3-1059804ea534" (UID: "f7d90120-9ee6-4ddb-92e3-1059804ea534"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 01:34:41.005455 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.005395    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-serving-cert" pod "f7d90120-9ee6-4ddb-92e3-1059804ea534" (UID: "f7d90120-9ee6-4ddb-92e3-1059804ea534"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 01:34:41.005455 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.005430    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7d90120-9ee6-4ddb-92e3-1059804ea534-kube-api-access-5r6hf" pod "f7d90120-9ee6-4ddb-92e3-1059804ea534" (UID: "f7d90120-9ee6-4ddb-92e3-1059804ea534"). InnerVolumeSpecName "kube-api-access-5r6hf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 01:34:41.104562 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.104531    4515 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-oauth-config\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:34:41.104562 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.104557    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5r6hf\" (UniqueName: \"kubernetes.io/projected/f7d90120-9ee6-4ddb-92e3-1059804ea534-kube-api-access-5r6hf\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:34:41.104736 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.104573    4515 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7d90120-9ee6-4ddb-92e3-1059804ea534-tmp\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:34:41.104736 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.104585    4515 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-serving-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:34:41.104736 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.104598    4515 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-console-config\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:34:41.104736 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.104610    4515 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-service-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:34:41.104736 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.104622    4515 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-oauth-serving-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:34:41.104736 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.104634    4515 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7d90120-9ee6-4ddb-92e3-1059804ea534-trusted-ca-bundle\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:34:41.560903 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.560871    4515 generic.go:356] "Generic (PLEG): container finished" podID="f7d90120-9ee6-4ddb-92e3-1059804ea534" containerID="ef1f75195138bc9d3208546528a1ddf1a253aec7a8937967faeb064b6a05b61c" exitCode=0
Nov 02 01:34:41.561060 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.560921    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d969c6fb7-42snk"
Nov 02 01:34:41.561060 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.560920    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d969c6fb7-42snk" event={"ID":"f7d90120-9ee6-4ddb-92e3-1059804ea534","Type":"ContainerDied","Data":"ef1f75195138bc9d3208546528a1ddf1a253aec7a8937967faeb064b6a05b61c"}
Nov 02 01:34:41.561060 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.561029    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d969c6fb7-42snk" event={"ID":"f7d90120-9ee6-4ddb-92e3-1059804ea534","Type":"ContainerDied","Data":"1375cb6fe8e80a5323ff72f76e013f4550e5c1899cec878f84844cc95d3e54bf"}
Nov 02 01:34:41.561060 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.561057    4515 scope.go:122] "RemoveContainer" containerID="ef1f75195138bc9d3208546528a1ddf1a253aec7a8937967faeb064b6a05b61c"
Nov 02 01:34:41.569339 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.569198    4515 scope.go:122] "RemoveContainer" containerID="ef1f75195138bc9d3208546528a1ddf1a253aec7a8937967faeb064b6a05b61c"
Nov 02 01:34:41.569568 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:34:41.569536    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef1f75195138bc9d3208546528a1ddf1a253aec7a8937967faeb064b6a05b61c\": container with ID starting with ef1f75195138bc9d3208546528a1ddf1a253aec7a8937967faeb064b6a05b61c not found: ID does not exist" containerID="ef1f75195138bc9d3208546528a1ddf1a253aec7a8937967faeb064b6a05b61c"
Nov 02 01:34:41.569677 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.569572    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef1f75195138bc9d3208546528a1ddf1a253aec7a8937967faeb064b6a05b61c"} err="failed to get container status \"ef1f75195138bc9d3208546528a1ddf1a253aec7a8937967faeb064b6a05b61c\": rpc error: code = NotFound desc = could not find container \"ef1f75195138bc9d3208546528a1ddf1a253aec7a8937967faeb064b6a05b61c\": container with ID starting with ef1f75195138bc9d3208546528a1ddf1a253aec7a8937967faeb064b6a05b61c not found: ID does not exist"
Nov 02 01:34:41.723857 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.723828    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d969c6fb7-42snk"]
Nov 02 01:34:41.773547 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:41.773518    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d969c6fb7-42snk"]
Nov 02 01:34:42.949984 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:34:42.949946    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f7d90120-9ee6-4ddb-92e3-1059804ea534" path="/var/lib/kubelet/pods/f7d90120-9ee6-4ddb-92e3-1059804ea534/volumes"
Nov 02 01:35:09.957083 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:35:09.957035    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:35:39.957805 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:35:39.957721    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:36:09.956770 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:36:09.956734    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:36:29.225421 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.225387    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k"]
Nov 02 01:36:29.227811 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.225617    4515 cpu_manager.go:414] "containerMap: removing container" podUID="f7d90120-9ee6-4ddb-92e3-1059804ea534" containerName="console"
Nov 02 01:36:29.227811 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.225629    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7d90120-9ee6-4ddb-92e3-1059804ea534" containerName="console"
Nov 02 01:36:29.227811 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.225680    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7d90120-9ee6-4ddb-92e3-1059804ea534" containerName="console"
Nov 02 01:36:29.228689 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.228673    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k"
Nov 02 01:36:29.330148 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.330103    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 02 01:36:29.330308 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.330175    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 02 01:36:29.330897 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.330876    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k"]
Nov 02 01:36:29.331356 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.331340    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71ad96b6-85ae-4d8a-946c-b9f27703fb23-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k\" (UID: \"71ad96b6-85ae-4d8a-946c-b9f27703fb23\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k"
Nov 02 01:36:29.331396 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.331381    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jkfq\" (UniqueName: \"kubernetes.io/projected/71ad96b6-85ae-4d8a-946c-b9f27703fb23-kube-api-access-2jkfq\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k\" (UID: \"71ad96b6-85ae-4d8a-946c-b9f27703fb23\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k"
Nov 02 01:36:29.331506 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.331492    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71ad96b6-85ae-4d8a-946c-b9f27703fb23-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k\" (UID: \"71ad96b6-85ae-4d8a-946c-b9f27703fb23\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k"
Nov 02 01:36:29.379698 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.379672    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-8tkpx\""
Nov 02 01:36:29.431996 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.431972    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71ad96b6-85ae-4d8a-946c-b9f27703fb23-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k\" (UID: \"71ad96b6-85ae-4d8a-946c-b9f27703fb23\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k"
Nov 02 01:36:29.432144 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.432005    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jkfq\" (UniqueName: \"kubernetes.io/projected/71ad96b6-85ae-4d8a-946c-b9f27703fb23-kube-api-access-2jkfq\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k\" (UID: \"71ad96b6-85ae-4d8a-946c-b9f27703fb23\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k"
Nov 02 01:36:29.432144 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.432050    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71ad96b6-85ae-4d8a-946c-b9f27703fb23-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k\" (UID: \"71ad96b6-85ae-4d8a-946c-b9f27703fb23\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k"
Nov 02 01:36:29.432354 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.432334    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71ad96b6-85ae-4d8a-946c-b9f27703fb23-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k\" (UID: \"71ad96b6-85ae-4d8a-946c-b9f27703fb23\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k"
Nov 02 01:36:29.432423 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.432372    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71ad96b6-85ae-4d8a-946c-b9f27703fb23-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k\" (UID: \"71ad96b6-85ae-4d8a-946c-b9f27703fb23\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k"
Nov 02 01:36:29.493074 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.493001    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jkfq\" (UniqueName: \"kubernetes.io/projected/71ad96b6-85ae-4d8a-946c-b9f27703fb23-kube-api-access-2jkfq\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k\" (UID: \"71ad96b6-85ae-4d8a-946c-b9f27703fb23\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k"
Nov 02 01:36:29.521260 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.521232    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"]
Nov 02 01:36:29.524197 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.524183    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.543389 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.543366    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k"
Nov 02 01:36:29.577096 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.577070    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 02 01:36:29.577232 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.577162    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 02 01:36:29.577278 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.577245    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 02 01:36:29.577734 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.577720    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 02 01:36:29.577783 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.577724    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 02 01:36:29.577818 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.577808    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 02 01:36:29.577901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.577888    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-f8tcq\""
Nov 02 01:36:29.585739 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.585712    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"]
Nov 02 01:36:29.632848 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.632822    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.633007 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.632860    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.633007 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.632884    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8aae44c3-965f-4d05-b70a-630b8758b93c-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.633007 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.632962    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.633136 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.633011    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.633136 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.633087    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.633136 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.633132    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thbhp\" (UniqueName: \"kubernetes.io/projected/8aae44c3-965f-4d05-b70a-630b8758b93c-kube-api-access-thbhp\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.734168 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.734133    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.734168 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.734170    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8aae44c3-965f-4d05-b70a-630b8758b93c-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.734399 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.734191    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.734399 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.734215    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.734399 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.734245    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.734399 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.734272    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thbhp\" (UniqueName: \"kubernetes.io/projected/8aae44c3-965f-4d05-b70a-630b8758b93c-kube-api-access-thbhp\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.734607 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.734417    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.734905 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.734869    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.736594 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.736567    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.736683 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.736604    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8aae44c3-965f-4d05-b70a-630b8758b93c-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.736683 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.736650    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.736683 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.736657    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.803634 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.803580    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.803754 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.803736    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thbhp\" (UniqueName: \"kubernetes.io/projected/8aae44c3-965f-4d05-b70a-630b8758b93c-kube-api-access-thbhp\") pod \"istiod-openshift-gateway-7876bfcb74-dgkjr\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.835255 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.835233    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:29.919895 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:29.919868    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k"]
Nov 02 01:36:29.923128 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:36:29.923075    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ad96b6_85ae_4d8a_946c_b9f27703fb23.slice/crio-0564654f01786c3cc5373f6b1d84ba5bff8e7c0b96c24a05f5c1afde256aeac4 WatchSource:0}: Error finding container 0564654f01786c3cc5373f6b1d84ba5bff8e7c0b96c24a05f5c1afde256aeac4: Status 404 returned error can't find the container with id 0564654f01786c3cc5373f6b1d84ba5bff8e7c0b96c24a05f5c1afde256aeac4
Nov 02 01:36:30.207212 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:30.207182    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"]
Nov 02 01:36:30.210898 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:36:30.210872    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aae44c3_965f_4d05_b70a_630b8758b93c.slice/crio-06b67ee5422490fef47ce1b3f358ffd7e3501a0d1bec74a0da7ed468dccdfbbb WatchSource:0}: Error finding container 06b67ee5422490fef47ce1b3f358ffd7e3501a0d1bec74a0da7ed468dccdfbbb: Status 404 returned error can't find the container with id 06b67ee5422490fef47ce1b3f358ffd7e3501a0d1bec74a0da7ed468dccdfbbb
Nov 02 01:36:30.889036 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:30.889005    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k" event={"ID":"71ad96b6-85ae-4d8a-946c-b9f27703fb23","Type":"ContainerStarted","Data":"0564654f01786c3cc5373f6b1d84ba5bff8e7c0b96c24a05f5c1afde256aeac4"}
Nov 02 01:36:30.890165 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:30.890140    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr" event={"ID":"8aae44c3-965f-4d05-b70a-630b8758b93c","Type":"ContainerStarted","Data":"06b67ee5422490fef47ce1b3f358ffd7e3501a0d1bec74a0da7ed468dccdfbbb"}
Nov 02 01:36:34.394995 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:34.394947    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:36:34.395383 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:34.395036    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:36:34.905933 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:34.905899    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr" event={"ID":"8aae44c3-965f-4d05-b70a-630b8758b93c","Type":"ContainerStarted","Data":"c0eff639cc9ce68807725fa5466808b77e10dc988bdd6e1df9d7db9f11389957"}
Nov 02 01:36:34.906146 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:34.906025    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:35.124331 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:35.124269    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr" podStartSLOduration=1.9433092090000001 podStartE2EDuration="6.124252737s" totalImagesPullingTime="4.180943528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:36:35.074510765 +0000 UTC m=+284.286880576" watchObservedRunningTime="2026-11-02 01:36:35.124252737 +0000 UTC m=+284.336622545"
Nov 02 01:36:35.912271 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:35.912228    4515 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-dgkjr container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 01:36:35.912700 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:35.912323    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr" podUID="8aae44c3-965f-4d05-b70a-630b8758b93c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 01:36:37.918387 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:37.918299    4515 generic.go:356] "Generic (PLEG): container finished" podID="71ad96b6-85ae-4d8a-946c-b9f27703fb23" containerID="c99361153b3a295fed08d99dda90990c92f394b99e9b7d37781ccdea3c1792ba" exitCode=0
Nov 02 01:36:37.918387 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:37.918365    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k" event={"ID":"71ad96b6-85ae-4d8a-946c-b9f27703fb23","Type":"ContainerDied","Data":"c99361153b3a295fed08d99dda90990c92f394b99e9b7d37781ccdea3c1792ba"}
Nov 02 01:36:38.913349 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:38.913327    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:36:39.927011 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:39.926969    4515 generic.go:356] "Generic (PLEG): container finished" podID="71ad96b6-85ae-4d8a-946c-b9f27703fb23" containerID="530a5d0dfe8faf7133adb6618704390d1be9686bfd71fd48968df36bec59d1ea" exitCode=0
Nov 02 01:36:39.927406 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:39.927065    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k" event={"ID":"71ad96b6-85ae-4d8a-946c-b9f27703fb23","Type":"ContainerDied","Data":"530a5d0dfe8faf7133adb6618704390d1be9686bfd71fd48968df36bec59d1ea"}
Nov 02 01:36:39.957710 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:36:39.957687    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:36:48.960814 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:48.960782    4515 generic.go:356] "Generic (PLEG): container finished" podID="71ad96b6-85ae-4d8a-946c-b9f27703fb23" containerID="3680f30dae9227fa3ea696c4143076b22dc7a3d5e66defc75bdf6b5940dfd422" exitCode=0
Nov 02 01:36:48.961227 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:48.960823    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k" event={"ID":"71ad96b6-85ae-4d8a-946c-b9f27703fb23","Type":"ContainerDied","Data":"3680f30dae9227fa3ea696c4143076b22dc7a3d5e66defc75bdf6b5940dfd422"}
Nov 02 01:36:50.080172 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:50.080142    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k"
Nov 02 01:36:50.203165 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:50.203136    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71ad96b6-85ae-4d8a-946c-b9f27703fb23-kube-api-access-2jkfq\" (UniqueName: \"kubernetes.io/projected/71ad96b6-85ae-4d8a-946c-b9f27703fb23-kube-api-access-2jkfq\") pod \"71ad96b6-85ae-4d8a-946c-b9f27703fb23\" (UID: \"71ad96b6-85ae-4d8a-946c-b9f27703fb23\") "
Nov 02 01:36:50.203325 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:50.203208    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71ad96b6-85ae-4d8a-946c-b9f27703fb23-bundle\" (UniqueName: \"kubernetes.io/empty-dir/71ad96b6-85ae-4d8a-946c-b9f27703fb23-bundle\") pod \"71ad96b6-85ae-4d8a-946c-b9f27703fb23\" (UID: \"71ad96b6-85ae-4d8a-946c-b9f27703fb23\") "
Nov 02 01:36:50.203325 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:50.203250    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71ad96b6-85ae-4d8a-946c-b9f27703fb23-util\" (UniqueName: \"kubernetes.io/empty-dir/71ad96b6-85ae-4d8a-946c-b9f27703fb23-util\") pod \"71ad96b6-85ae-4d8a-946c-b9f27703fb23\" (UID: \"71ad96b6-85ae-4d8a-946c-b9f27703fb23\") "
Nov 02 01:36:50.203779 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:50.203751    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71ad96b6-85ae-4d8a-946c-b9f27703fb23-bundle" pod "71ad96b6-85ae-4d8a-946c-b9f27703fb23" (UID: "71ad96b6-85ae-4d8a-946c-b9f27703fb23"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:36:50.205323 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:50.205293    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71ad96b6-85ae-4d8a-946c-b9f27703fb23-kube-api-access-2jkfq" pod "71ad96b6-85ae-4d8a-946c-b9f27703fb23" (UID: "71ad96b6-85ae-4d8a-946c-b9f27703fb23"). InnerVolumeSpecName "kube-api-access-2jkfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 01:36:50.210288 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:50.210258    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71ad96b6-85ae-4d8a-946c-b9f27703fb23-util" pod "71ad96b6-85ae-4d8a-946c-b9f27703fb23" (UID: "71ad96b6-85ae-4d8a-946c-b9f27703fb23"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:36:50.303801 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:50.303739    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jkfq\" (UniqueName: \"kubernetes.io/projected/71ad96b6-85ae-4d8a-946c-b9f27703fb23-kube-api-access-2jkfq\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:36:50.303801 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:50.303763    4515 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71ad96b6-85ae-4d8a-946c-b9f27703fb23-bundle\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:36:50.303801 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:50.303773    4515 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71ad96b6-85ae-4d8a-946c-b9f27703fb23-util\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:36:50.969706 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:50.969681    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k"
Nov 02 01:36:50.969850 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:50.969681    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7189c8k" event={"ID":"71ad96b6-85ae-4d8a-946c-b9f27703fb23","Type":"ContainerDied","Data":"0564654f01786c3cc5373f6b1d84ba5bff8e7c0b96c24a05f5c1afde256aeac4"}
Nov 02 01:36:50.969850 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:50.969811    4515 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0564654f01786c3cc5373f6b1d84ba5bff8e7c0b96c24a05f5c1afde256aeac4"
Nov 02 01:36:50.977285 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:50.977260    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 01:36:50.977392 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:36:50.977326    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 01:37:09.957390 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:37:09.957311    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:37:39.959285 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:37:39.959229    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:37:43.528375 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.528341    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"]
Nov 02 01:37:43.528765 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.528576    4515 cpu_manager.go:414] "containerMap: removing container" podUID="71ad96b6-85ae-4d8a-946c-b9f27703fb23" containerName="pull"
Nov 02 01:37:43.528765 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.528587    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71ad96b6-85ae-4d8a-946c-b9f27703fb23" containerName="pull"
Nov 02 01:37:43.528765 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.528595    4515 cpu_manager.go:414] "containerMap: removing container" podUID="71ad96b6-85ae-4d8a-946c-b9f27703fb23" containerName="extract"
Nov 02 01:37:43.528765 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.528600    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71ad96b6-85ae-4d8a-946c-b9f27703fb23" containerName="extract"
Nov 02 01:37:43.528765 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.528624    4515 cpu_manager.go:414] "containerMap: removing container" podUID="71ad96b6-85ae-4d8a-946c-b9f27703fb23" containerName="util"
Nov 02 01:37:43.528765 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.528629    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71ad96b6-85ae-4d8a-946c-b9f27703fb23" containerName="util"
Nov 02 01:37:43.528765 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.528671    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="71ad96b6-85ae-4d8a-946c-b9f27703fb23" containerName="extract"
Nov 02 01:37:43.530233 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.530217    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.566993 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.566963    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b94ee5bd-42bd-490b-b350-17d4a9018112-webhook-cert\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.566993 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.567004    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b94ee5bd-42bd-490b-b350-17d4a9018112-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.567227 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.567028    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b94ee5bd-42bd-490b-b350-17d4a9018112-manifests\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.567227 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.567077    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7pzj\" (UniqueName: \"kubernetes.io/projected/b94ee5bd-42bd-490b-b350-17d4a9018112-kube-api-access-p7pzj\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.567227 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.567179    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b94ee5bd-42bd-490b-b350-17d4a9018112-metrics-cert\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.567227 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.567206    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b94ee5bd-42bd-490b-b350-17d4a9018112-tmp\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.603293 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.603266    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 02 01:37:43.603293 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.603290    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 02 01:37:43.633454 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.633423    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 02 01:37:43.633624 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.633486    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 02 01:37:43.633624 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.633508    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-gnd7s\""
Nov 02 01:37:43.633624 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.633593    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 02 01:37:43.634746 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.634730    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"]
Nov 02 01:37:43.667635 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.667604    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b94ee5bd-42bd-490b-b350-17d4a9018112-tmp\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.667827 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.667763    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b94ee5bd-42bd-490b-b350-17d4a9018112-webhook-cert\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.667827 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.667812    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b94ee5bd-42bd-490b-b350-17d4a9018112-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.668011 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.667842    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b94ee5bd-42bd-490b-b350-17d4a9018112-manifests\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.668011 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.667880    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7pzj\" (UniqueName: \"kubernetes.io/projected/b94ee5bd-42bd-490b-b350-17d4a9018112-kube-api-access-p7pzj\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.668011 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.667918    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b94ee5bd-42bd-490b-b350-17d4a9018112-metrics-cert\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.668449 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.668424    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b94ee5bd-42bd-490b-b350-17d4a9018112-manifests\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.670353 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.670331    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b94ee5bd-42bd-490b-b350-17d4a9018112-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.670446 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.670360    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b94ee5bd-42bd-490b-b350-17d4a9018112-metrics-cert\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.670446 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.670422    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b94ee5bd-42bd-490b-b350-17d4a9018112-tmp\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.670554 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.670537    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b94ee5bd-42bd-490b-b350-17d4a9018112-webhook-cert\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.729034 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.728997    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7pzj\" (UniqueName: \"kubernetes.io/projected/b94ee5bd-42bd-490b-b350-17d4a9018112-kube-api-access-p7pzj\") pod \"opendatahub-operator-controller-manager-6755fff5c-dqgg5\" (UID: \"b94ee5bd-42bd-490b-b350-17d4a9018112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:43.843940 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:43.843848    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:44.358737 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:44.358705    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"]
Nov 02 01:37:44.361998 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:37:44.361958    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb94ee5bd_42bd_490b_b350_17d4a9018112.slice/crio-c66b031ae4232631738d6b356fc4cff6454bf3b66d886dcffae11e93a8237728 WatchSource:0}: Error finding container c66b031ae4232631738d6b356fc4cff6454bf3b66d886dcffae11e93a8237728: Status 404 returned error can't find the container with id c66b031ae4232631738d6b356fc4cff6454bf3b66d886dcffae11e93a8237728
Nov 02 01:37:44.363607 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:44.363590    4515 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 01:37:45.137547 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:45.137515    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5" event={"ID":"b94ee5bd-42bd-490b-b350-17d4a9018112","Type":"ContainerStarted","Data":"c66b031ae4232631738d6b356fc4cff6454bf3b66d886dcffae11e93a8237728"}
Nov 02 01:37:48.150468 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:48.150391    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5" event={"ID":"b94ee5bd-42bd-490b-b350-17d4a9018112","Type":"ContainerStarted","Data":"9558245cdabab738ffd80ff69151e960508bb0a3e69427a38a24d3fd00f10f88"}
Nov 02 01:37:49.155312 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:49.155276    4515 generic.go:356] "Generic (PLEG): container finished" podID="b94ee5bd-42bd-490b-b350-17d4a9018112" containerID="9558245cdabab738ffd80ff69151e960508bb0a3e69427a38a24d3fd00f10f88" exitCode=0
Nov 02 01:37:49.155758 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:49.155351    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5" event={"ID":"b94ee5bd-42bd-490b-b350-17d4a9018112","Type":"ContainerDied","Data":"9558245cdabab738ffd80ff69151e960508bb0a3e69427a38a24d3fd00f10f88"}
Nov 02 01:37:50.160759 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:50.160718    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5" event={"ID":"b94ee5bd-42bd-490b-b350-17d4a9018112","Type":"ContainerStarted","Data":"cade62699f00318b3695d149763336b6ad38600b9dc210789008a98c7d8173a1"}
Nov 02 01:37:50.161189 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:50.160837    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:37:50.419674 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:37:50.419572    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5" podStartSLOduration=2.800826261 podStartE2EDuration="7.419557068s" totalImagesPullingTime="4.171892603s" totalInitContainerRuntime="446.838204ms" isStatefulPod=false podCreationTimestamp="2026-11-02 01:37:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:37:50.369982402 +0000 UTC m=+359.582352210" watchObservedRunningTime="2026-11-02 01:37:50.419557068 +0000 UTC m=+359.631926875"
Nov 02 01:38:01.166614 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:01.166585    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6755fff5c-dqgg5"
Nov 02 01:38:09.958218 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:38:09.958173    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:38:14.434546 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:14.434507    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-bf99ddbdb-jvtcn"]
Nov 02 01:38:14.438707 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:14.438683    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-bf99ddbdb-jvtcn"
Nov 02 01:38:14.516593 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:14.516562    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-bf99ddbdb-jvtcn"]
Nov 02 01:38:14.540801 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:14.540771    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 02 01:38:14.540957 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:14.540828    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9jznf\""
Nov 02 01:38:14.540957 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:14.540929    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 02 01:38:14.604842 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:14.604807    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/909a5737-94eb-413e-87d7-9cd901ed51b2-tls-certs\") pod \"kube-auth-proxy-bf99ddbdb-jvtcn\" (UID: \"909a5737-94eb-413e-87d7-9cd901ed51b2\") " pod="openshift-ingress/kube-auth-proxy-bf99ddbdb-jvtcn"
Nov 02 01:38:14.604977 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:14.604865    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7b4b\" (UniqueName: \"kubernetes.io/projected/909a5737-94eb-413e-87d7-9cd901ed51b2-kube-api-access-b7b4b\") pod \"kube-auth-proxy-bf99ddbdb-jvtcn\" (UID: \"909a5737-94eb-413e-87d7-9cd901ed51b2\") " pod="openshift-ingress/kube-auth-proxy-bf99ddbdb-jvtcn"
Nov 02 01:38:14.605019 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:14.604974    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/909a5737-94eb-413e-87d7-9cd901ed51b2-tmp\") pod \"kube-auth-proxy-bf99ddbdb-jvtcn\" (UID: \"909a5737-94eb-413e-87d7-9cd901ed51b2\") " pod="openshift-ingress/kube-auth-proxy-bf99ddbdb-jvtcn"
Nov 02 01:38:14.705806 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:14.705776    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/909a5737-94eb-413e-87d7-9cd901ed51b2-tls-certs\") pod \"kube-auth-proxy-bf99ddbdb-jvtcn\" (UID: \"909a5737-94eb-413e-87d7-9cd901ed51b2\") " pod="openshift-ingress/kube-auth-proxy-bf99ddbdb-jvtcn"
Nov 02 01:38:14.705972 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:14.705831    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7b4b\" (UniqueName: \"kubernetes.io/projected/909a5737-94eb-413e-87d7-9cd901ed51b2-kube-api-access-b7b4b\") pod \"kube-auth-proxy-bf99ddbdb-jvtcn\" (UID: \"909a5737-94eb-413e-87d7-9cd901ed51b2\") " pod="openshift-ingress/kube-auth-proxy-bf99ddbdb-jvtcn"
Nov 02 01:38:14.706038 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:14.705961    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/909a5737-94eb-413e-87d7-9cd901ed51b2-tmp\") pod \"kube-auth-proxy-bf99ddbdb-jvtcn\" (UID: \"909a5737-94eb-413e-87d7-9cd901ed51b2\") " pod="openshift-ingress/kube-auth-proxy-bf99ddbdb-jvtcn"
Nov 02 01:38:14.708110 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:14.708087    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/909a5737-94eb-413e-87d7-9cd901ed51b2-tmp\") pod \"kube-auth-proxy-bf99ddbdb-jvtcn\" (UID: \"909a5737-94eb-413e-87d7-9cd901ed51b2\") " pod="openshift-ingress/kube-auth-proxy-bf99ddbdb-jvtcn"
Nov 02 01:38:14.708329 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:14.708310    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/909a5737-94eb-413e-87d7-9cd901ed51b2-tls-certs\") pod \"kube-auth-proxy-bf99ddbdb-jvtcn\" (UID: \"909a5737-94eb-413e-87d7-9cd901ed51b2\") " pod="openshift-ingress/kube-auth-proxy-bf99ddbdb-jvtcn"
Nov 02 01:38:14.765880 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:14.765848    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7b4b\" (UniqueName: \"kubernetes.io/projected/909a5737-94eb-413e-87d7-9cd901ed51b2-kube-api-access-b7b4b\") pod \"kube-auth-proxy-bf99ddbdb-jvtcn\" (UID: \"909a5737-94eb-413e-87d7-9cd901ed51b2\") " pod="openshift-ingress/kube-auth-proxy-bf99ddbdb-jvtcn"
Nov 02 01:38:15.048638 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:15.048555    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-bf99ddbdb-jvtcn"
Nov 02 01:38:15.417068 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:15.416989    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-bf99ddbdb-jvtcn"]
Nov 02 01:38:15.420774 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:38:15.420748    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod909a5737_94eb_413e_87d7_9cd901ed51b2.slice/crio-0d59199a0fbabede6fa09c25348c9bc76c9b242b05c82be87988489599a8c63b WatchSource:0}: Error finding container 0d59199a0fbabede6fa09c25348c9bc76c9b242b05c82be87988489599a8c63b: Status 404 returned error can't find the container with id 0d59199a0fbabede6fa09c25348c9bc76c9b242b05c82be87988489599a8c63b
Nov 02 01:38:16.250650 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:16.250616    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-bf99ddbdb-jvtcn" event={"ID":"909a5737-94eb-413e-87d7-9cd901ed51b2","Type":"ContainerStarted","Data":"0d59199a0fbabede6fa09c25348c9bc76c9b242b05c82be87988489599a8c63b"}
Nov 02 01:38:19.263254 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:19.263224    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-bf99ddbdb-jvtcn" event={"ID":"909a5737-94eb-413e-87d7-9cd901ed51b2","Type":"ContainerStarted","Data":"45f6a0e504c0e805ac7eeef72e8e1f54602cfe612ca7c1ffdcd0bd5046803446"}
Nov 02 01:38:19.378587 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:19.378457    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-bf99ddbdb-jvtcn" podStartSLOduration=1.688160565 podStartE2EDuration="5.378442513s" totalImagesPullingTime="3.690281948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:38:19.378183311 +0000 UTC m=+388.590553121" watchObservedRunningTime="2026-11-02 01:38:19.378442513 +0000 UTC m=+388.590812321"
Nov 02 01:38:27.377062 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.377031    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"]
Nov 02 01:38:27.380210 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.380108    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"
Nov 02 01:38:27.482026 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.481969    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Nov 02 01:38:27.482026 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.481997    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 02 01:38:27.482026 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.482006    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Nov 02 01:38:27.482026 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.482022    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 02 01:38:27.482409 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.482163    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Nov 02 01:38:27.482409 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.482187    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-krcbf\""
Nov 02 01:38:27.483503 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.483482    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"]
Nov 02 01:38:27.500449 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.500415    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlkdf\" (UniqueName: \"kubernetes.io/projected/0674773d-c62f-4825-859a-fb12a69d0b97-kube-api-access-dlkdf\") pod \"jobset-controller-manager-7c88975c69-wgsd9\" (UID: \"0674773d-c62f-4825-859a-fb12a69d0b97\") " pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"
Nov 02 01:38:27.500617 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.500471    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0674773d-c62f-4825-859a-fb12a69d0b97-metrics-certs\") pod \"jobset-controller-manager-7c88975c69-wgsd9\" (UID: \"0674773d-c62f-4825-859a-fb12a69d0b97\") " pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"
Nov 02 01:38:27.500617 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.500529    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0674773d-c62f-4825-859a-fb12a69d0b97-manager-config\") pod \"jobset-controller-manager-7c88975c69-wgsd9\" (UID: \"0674773d-c62f-4825-859a-fb12a69d0b97\") " pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"
Nov 02 01:38:27.500617 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.500574    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0674773d-c62f-4825-859a-fb12a69d0b97-cert\") pod \"jobset-controller-manager-7c88975c69-wgsd9\" (UID: \"0674773d-c62f-4825-859a-fb12a69d0b97\") " pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"
Nov 02 01:38:27.600998 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.600959    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0674773d-c62f-4825-859a-fb12a69d0b97-cert\") pod \"jobset-controller-manager-7c88975c69-wgsd9\" (UID: \"0674773d-c62f-4825-859a-fb12a69d0b97\") " pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"
Nov 02 01:38:27.601185 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.601042    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlkdf\" (UniqueName: \"kubernetes.io/projected/0674773d-c62f-4825-859a-fb12a69d0b97-kube-api-access-dlkdf\") pod \"jobset-controller-manager-7c88975c69-wgsd9\" (UID: \"0674773d-c62f-4825-859a-fb12a69d0b97\") " pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"
Nov 02 01:38:27.601185 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.601094    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0674773d-c62f-4825-859a-fb12a69d0b97-metrics-certs\") pod \"jobset-controller-manager-7c88975c69-wgsd9\" (UID: \"0674773d-c62f-4825-859a-fb12a69d0b97\") " pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"
Nov 02 01:38:27.601185 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.601151    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0674773d-c62f-4825-859a-fb12a69d0b97-manager-config\") pod \"jobset-controller-manager-7c88975c69-wgsd9\" (UID: \"0674773d-c62f-4825-859a-fb12a69d0b97\") " pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"
Nov 02 01:38:27.601840 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.601814    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0674773d-c62f-4825-859a-fb12a69d0b97-manager-config\") pod \"jobset-controller-manager-7c88975c69-wgsd9\" (UID: \"0674773d-c62f-4825-859a-fb12a69d0b97\") " pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"
Nov 02 01:38:27.603747 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.603729    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0674773d-c62f-4825-859a-fb12a69d0b97-cert\") pod \"jobset-controller-manager-7c88975c69-wgsd9\" (UID: \"0674773d-c62f-4825-859a-fb12a69d0b97\") " pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"
Nov 02 01:38:27.603831 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.603786    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0674773d-c62f-4825-859a-fb12a69d0b97-metrics-certs\") pod \"jobset-controller-manager-7c88975c69-wgsd9\" (UID: \"0674773d-c62f-4825-859a-fb12a69d0b97\") " pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"
Nov 02 01:38:27.667639 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.667565    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlkdf\" (UniqueName: \"kubernetes.io/projected/0674773d-c62f-4825-859a-fb12a69d0b97-kube-api-access-dlkdf\") pod \"jobset-controller-manager-7c88975c69-wgsd9\" (UID: \"0674773d-c62f-4825-859a-fb12a69d0b97\") " pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"
Nov 02 01:38:27.691470 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:27.691435    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"
Nov 02 01:38:28.060731 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:28.060688    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"]
Nov 02 01:38:28.064252 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:38:28.064230    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0674773d_c62f_4825_859a_fb12a69d0b97.slice/crio-d8e83e12f08d54410ba60bb01aa71416db26dbe89df18b5e7a93d4ff6df59982 WatchSource:0}: Error finding container d8e83e12f08d54410ba60bb01aa71416db26dbe89df18b5e7a93d4ff6df59982: Status 404 returned error can't find the container with id d8e83e12f08d54410ba60bb01aa71416db26dbe89df18b5e7a93d4ff6df59982
Nov 02 01:38:28.298253 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:28.298211    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9" event={"ID":"0674773d-c62f-4825-859a-fb12a69d0b97","Type":"ContainerStarted","Data":"d8e83e12f08d54410ba60bb01aa71416db26dbe89df18b5e7a93d4ff6df59982"}
Nov 02 01:38:34.321943 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:34.321907    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9" event={"ID":"0674773d-c62f-4825-859a-fb12a69d0b97","Type":"ContainerStarted","Data":"a5b427ab1947f865721f905b55190f19c0911cdec9271fddac959946a360c3f9"}
Nov 02 01:38:34.322347 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:34.322035    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"
Nov 02 01:38:34.437634 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:34.437577    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9" podStartSLOduration=2.105477238 podStartE2EDuration="7.437563264s" totalImagesPullingTime="5.332086026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:38:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:38:34.437516447 +0000 UTC m=+403.649886258" watchObservedRunningTime="2026-11-02 01:38:34.437563264 +0000 UTC m=+403.649933073"
Nov 02 01:38:39.958899 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:38:39.958859    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:38:45.330864 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:38:45.330833    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-7c88975c69-wgsd9"
Nov 02 01:39:09.957588 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:39:09.957553    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:39:35.002240 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.002208    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-qtvjg"]
Nov 02 01:39:35.004589 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.004573    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-qtvjg"
Nov 02 01:39:35.056692 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.056665    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 02 01:39:35.106204 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.106176    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 02 01:39:35.107244 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.107222    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-qtvjg"]
Nov 02 01:39:35.109763 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.109743    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Nov 02 01:39:35.109878 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.109770    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-9lq2q\""
Nov 02 01:39:35.117102 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.117075    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6a91787f-612d-40cc-96ab-af1200465e9c-cert\") pod \"odh-model-controller-769875db56-qtvjg\" (UID: \"6a91787f-612d-40cc-96ab-af1200465e9c\") " pod="opendatahub/odh-model-controller-769875db56-qtvjg"
Nov 02 01:39:35.117240 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.117166    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a91787f-612d-40cc-96ab-af1200465e9c-metrics-certs\") pod \"odh-model-controller-769875db56-qtvjg\" (UID: \"6a91787f-612d-40cc-96ab-af1200465e9c\") " pod="opendatahub/odh-model-controller-769875db56-qtvjg"
Nov 02 01:39:35.117240 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.117212    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rdsw\" (UniqueName: \"kubernetes.io/projected/6a91787f-612d-40cc-96ab-af1200465e9c-kube-api-access-8rdsw\") pod \"odh-model-controller-769875db56-qtvjg\" (UID: \"6a91787f-612d-40cc-96ab-af1200465e9c\") " pod="opendatahub/odh-model-controller-769875db56-qtvjg"
Nov 02 01:39:35.155867 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.155839    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Nov 02 01:39:35.218025 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.217992    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a91787f-612d-40cc-96ab-af1200465e9c-metrics-certs\") pod \"odh-model-controller-769875db56-qtvjg\" (UID: \"6a91787f-612d-40cc-96ab-af1200465e9c\") " pod="opendatahub/odh-model-controller-769875db56-qtvjg"
Nov 02 01:39:35.218231 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.218036    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rdsw\" (UniqueName: \"kubernetes.io/projected/6a91787f-612d-40cc-96ab-af1200465e9c-kube-api-access-8rdsw\") pod \"odh-model-controller-769875db56-qtvjg\" (UID: \"6a91787f-612d-40cc-96ab-af1200465e9c\") " pod="opendatahub/odh-model-controller-769875db56-qtvjg"
Nov 02 01:39:35.218231 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.218079    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6a91787f-612d-40cc-96ab-af1200465e9c-cert\") pod \"odh-model-controller-769875db56-qtvjg\" (UID: \"6a91787f-612d-40cc-96ab-af1200465e9c\") " pod="opendatahub/odh-model-controller-769875db56-qtvjg"
Nov 02 01:39:35.220399 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.220375    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a91787f-612d-40cc-96ab-af1200465e9c-metrics-certs\") pod \"odh-model-controller-769875db56-qtvjg\" (UID: \"6a91787f-612d-40cc-96ab-af1200465e9c\") " pod="opendatahub/odh-model-controller-769875db56-qtvjg"
Nov 02 01:39:35.220499 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.220447    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6a91787f-612d-40cc-96ab-af1200465e9c-cert\") pod \"odh-model-controller-769875db56-qtvjg\" (UID: \"6a91787f-612d-40cc-96ab-af1200465e9c\") " pod="opendatahub/odh-model-controller-769875db56-qtvjg"
Nov 02 01:39:35.285807 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.285745    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rdsw\" (UniqueName: \"kubernetes.io/projected/6a91787f-612d-40cc-96ab-af1200465e9c-kube-api-access-8rdsw\") pod \"odh-model-controller-769875db56-qtvjg\" (UID: \"6a91787f-612d-40cc-96ab-af1200465e9c\") " pod="opendatahub/odh-model-controller-769875db56-qtvjg"
Nov 02 01:39:35.316580 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.316549    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-qtvjg"
Nov 02 01:39:35.686055 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:35.686027    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-qtvjg"]
Nov 02 01:39:35.689596 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:39:35.689571    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a91787f_612d_40cc_96ab_af1200465e9c.slice/crio-5e07f2e05a6598ca2c52097e4eba0107c3245c067eab62b5441dcde5dc2802d7 WatchSource:0}: Error finding container 5e07f2e05a6598ca2c52097e4eba0107c3245c067eab62b5441dcde5dc2802d7: Status 404 returned error can't find the container with id 5e07f2e05a6598ca2c52097e4eba0107c3245c067eab62b5441dcde5dc2802d7
Nov 02 01:39:36.539105 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:36.539070    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-qtvjg" event={"ID":"6a91787f-612d-40cc-96ab-af1200465e9c","Type":"ContainerStarted","Data":"5e07f2e05a6598ca2c52097e4eba0107c3245c067eab62b5441dcde5dc2802d7"}
Nov 02 01:39:37.766564 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:37.766531    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"]
Nov 02 01:39:37.766999 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:37.766715    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr" podUID="8aae44c3-965f-4d05-b70a-630b8758b93c" containerName="discovery" containerID="cri-o://c0eff639cc9ce68807725fa5466808b77e10dc988bdd6e1df9d7db9f11389957" gracePeriod=30
Nov 02 01:39:38.024335 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.024271    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:39:38.143310 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.143277    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-kubeconfig\") pod \"8aae44c3-965f-4d05-b70a-630b8758b93c\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") "
Nov 02 01:39:38.143310 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.143316    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-token\" (UniqueName: \"kubernetes.io/projected/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-token\") pod \"8aae44c3-965f-4d05-b70a-630b8758b93c\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") "
Nov 02 01:39:38.143553 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.143344    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8aae44c3-965f-4d05-b70a-630b8758b93c-kube-api-access-thbhp\" (UniqueName: \"kubernetes.io/projected/8aae44c3-965f-4d05-b70a-630b8758b93c-kube-api-access-thbhp\") pod \"8aae44c3-965f-4d05-b70a-630b8758b93c\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") "
Nov 02 01:39:38.143553 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.143374    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8aae44c3-965f-4d05-b70a-630b8758b93c-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8aae44c3-965f-4d05-b70a-630b8758b93c-local-certs\") pod \"8aae44c3-965f-4d05-b70a-630b8758b93c\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") "
Nov 02 01:39:38.143553 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.143453    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-csr-dns-cert\") pod \"8aae44c3-965f-4d05-b70a-630b8758b93c\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") "
Nov 02 01:39:38.143553 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.143478    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-cacerts\" (UniqueName: \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-cacerts\") pod \"8aae44c3-965f-4d05-b70a-630b8758b93c\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") "
Nov 02 01:39:38.143553 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.143529    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-csr-ca-configmap\") pod \"8aae44c3-965f-4d05-b70a-630b8758b93c\" (UID: \"8aae44c3-965f-4d05-b70a-630b8758b93c\") "
Nov 02 01:39:38.144080 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.144027    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-csr-ca-configmap" pod "8aae44c3-965f-4d05-b70a-630b8758b93c" (UID: "8aae44c3-965f-4d05-b70a-630b8758b93c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:39:38.147368 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.147052    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-csr-dns-cert" pod "8aae44c3-965f-4d05-b70a-630b8758b93c" (UID: "8aae44c3-965f-4d05-b70a-630b8758b93c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 01:39:38.147368 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.147299    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aae44c3-965f-4d05-b70a-630b8758b93c-kube-api-access-thbhp" pod "8aae44c3-965f-4d05-b70a-630b8758b93c" (UID: "8aae44c3-965f-4d05-b70a-630b8758b93c"). InnerVolumeSpecName "kube-api-access-thbhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 01:39:38.147525 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.147387    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aae44c3-965f-4d05-b70a-630b8758b93c-local-certs" pod "8aae44c3-965f-4d05-b70a-630b8758b93c" (UID: "8aae44c3-965f-4d05-b70a-630b8758b93c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:39:38.147620 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.147594    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-cacerts" pod "8aae44c3-965f-4d05-b70a-630b8758b93c" (UID: "8aae44c3-965f-4d05-b70a-630b8758b93c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 01:39:38.147849 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.147832    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-token" pod "8aae44c3-965f-4d05-b70a-630b8758b93c" (UID: "8aae44c3-965f-4d05-b70a-630b8758b93c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 01:39:38.148381 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.148356    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-kubeconfig" pod "8aae44c3-965f-4d05-b70a-630b8758b93c" (UID: "8aae44c3-965f-4d05-b70a-630b8758b93c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 01:39:38.244238 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.244205    4515 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-csr-ca-configmap\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:39:38.244238 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.244237    4515 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-kubeconfig\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:39:38.244461 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.244253    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:39:38.244461 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.244264    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thbhp\" (UniqueName: \"kubernetes.io/projected/8aae44c3-965f-4d05-b70a-630b8758b93c-kube-api-access-thbhp\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:39:38.244461 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.244273    4515 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8aae44c3-965f-4d05-b70a-630b8758b93c-local-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:39:38.244461 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.244281    4515 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-istio-csr-dns-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:39:38.244461 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.244290    4515 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8aae44c3-965f-4d05-b70a-630b8758b93c-cacerts\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:39:38.548687 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.548644    4515 generic.go:356] "Generic (PLEG): container finished" podID="8aae44c3-965f-4d05-b70a-630b8758b93c" containerID="c0eff639cc9ce68807725fa5466808b77e10dc988bdd6e1df9d7db9f11389957" exitCode=0
Nov 02 01:39:38.548875 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.548704    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"
Nov 02 01:39:38.548875 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.548732    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr" event={"ID":"8aae44c3-965f-4d05-b70a-630b8758b93c","Type":"ContainerDied","Data":"c0eff639cc9ce68807725fa5466808b77e10dc988bdd6e1df9d7db9f11389957"}
Nov 02 01:39:38.548875 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.548795    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr" event={"ID":"8aae44c3-965f-4d05-b70a-630b8758b93c","Type":"ContainerDied","Data":"06b67ee5422490fef47ce1b3f358ffd7e3501a0d1bec74a0da7ed468dccdfbbb"}
Nov 02 01:39:38.548875 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.548822    4515 scope.go:122] "RemoveContainer" containerID="c0eff639cc9ce68807725fa5466808b77e10dc988bdd6e1df9d7db9f11389957"
Nov 02 01:39:38.563404 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.563380    4515 scope.go:122] "RemoveContainer" containerID="c0eff639cc9ce68807725fa5466808b77e10dc988bdd6e1df9d7db9f11389957"
Nov 02 01:39:38.563712 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:39:38.563687    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0eff639cc9ce68807725fa5466808b77e10dc988bdd6e1df9d7db9f11389957\": container with ID starting with c0eff639cc9ce68807725fa5466808b77e10dc988bdd6e1df9d7db9f11389957 not found: ID does not exist" containerID="c0eff639cc9ce68807725fa5466808b77e10dc988bdd6e1df9d7db9f11389957"
Nov 02 01:39:38.563775 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.563724    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0eff639cc9ce68807725fa5466808b77e10dc988bdd6e1df9d7db9f11389957"} err="failed to get container status \"c0eff639cc9ce68807725fa5466808b77e10dc988bdd6e1df9d7db9f11389957\": rpc error: code = NotFound desc = could not find container \"c0eff639cc9ce68807725fa5466808b77e10dc988bdd6e1df9d7db9f11389957\": container with ID starting with c0eff639cc9ce68807725fa5466808b77e10dc988bdd6e1df9d7db9f11389957 not found: ID does not exist"
Nov 02 01:39:38.712664 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.712629    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"]
Nov 02 01:39:38.716785 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.716766    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-dgkjr"]
Nov 02 01:39:38.951183 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:38.951142    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8aae44c3-965f-4d05-b70a-630b8758b93c" path="/var/lib/kubelet/pods/8aae44c3-965f-4d05-b70a-630b8758b93c/volumes"
Nov 02 01:39:39.957802 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:39:39.957762    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:39:40.558977 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:40.558940    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-qtvjg" event={"ID":"6a91787f-612d-40cc-96ab-af1200465e9c","Type":"ContainerStarted","Data":"a157363f4eb13cbe6e0b76b0ac241a4990d4d82a73f14f5b83536dc6ad72423b"}
Nov 02 01:39:40.559159 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:40.559069    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-qtvjg"
Nov 02 01:39:40.675591 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:40.675529    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-qtvjg" podStartSLOduration=2.63893165 podStartE2EDuration="6.675510674s" totalImagesPullingTime="4.036579024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:39:40.674468199 +0000 UTC m=+469.886838008" watchObservedRunningTime="2026-11-02 01:39:40.675510674 +0000 UTC m=+469.887880483"
Nov 02 01:39:51.565210 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:51.565182    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-qtvjg"
Nov 02 01:39:52.999424 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:52.999386    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"]
Nov 02 01:39:52.999787 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:52.999638    4515 cpu_manager.go:414] "containerMap: removing container" podUID="8aae44c3-965f-4d05-b70a-630b8758b93c" containerName="discovery"
Nov 02 01:39:52.999787 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:52.999648    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8aae44c3-965f-4d05-b70a-630b8758b93c" containerName="discovery"
Nov 02 01:39:52.999787 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:52.999712    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="8aae44c3-965f-4d05-b70a-630b8758b93c" containerName="discovery"
Nov 02 01:39:53.006417 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.006400    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:39:53.069148 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.069100    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"]
Nov 02 01:39:53.109747 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.109716    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 02 01:39:53.109880 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.109774    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 02 01:39:53.109880 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.109802    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-2n4c5\""
Nov 02 01:39:53.109962 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.109939    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 02 01:39:53.110029 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.110018    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 02 01:39:53.158321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.158293    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-metrics-certs\") pod \"maas-api-6d669c7fcd-42pnt\" (UID: \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:39:53.158321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.158323    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-openshift-service-ca\") pod \"maas-api-6d669c7fcd-42pnt\" (UID: \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:39:53.158493 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.158346    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-maas-api-tls\") pod \"maas-api-6d669c7fcd-42pnt\" (UID: \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:39:53.158493 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.158365    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7742\" (UniqueName: \"kubernetes.io/projected/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-kube-api-access-v7742\") pod \"maas-api-6d669c7fcd-42pnt\" (UID: \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:39:53.259595 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.259527    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-metrics-certs\") pod \"maas-api-6d669c7fcd-42pnt\" (UID: \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:39:53.259595 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.259562    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-openshift-service-ca\") pod \"maas-api-6d669c7fcd-42pnt\" (UID: \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:39:53.259595 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.259590    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-maas-api-tls\") pod \"maas-api-6d669c7fcd-42pnt\" (UID: \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:39:53.259850 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.259611    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7742\" (UniqueName: \"kubernetes.io/projected/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-kube-api-access-v7742\") pod \"maas-api-6d669c7fcd-42pnt\" (UID: \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:39:53.260209 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.260191    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-openshift-service-ca\") pod \"maas-api-6d669c7fcd-42pnt\" (UID: \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:39:53.261934 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.261906    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-metrics-certs\") pod \"maas-api-6d669c7fcd-42pnt\" (UID: \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:39:53.262099 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.262075    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-maas-api-tls\") pod \"maas-api-6d669c7fcd-42pnt\" (UID: \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:39:53.319884 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.319858    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7742\" (UniqueName: \"kubernetes.io/projected/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-kube-api-access-v7742\") pod \"maas-api-6d669c7fcd-42pnt\" (UID: \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:39:53.618574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.618502    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:39:53.987658 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:53.987630    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"]
Nov 02 01:39:53.990980 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:39:53.990952    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e16e0a_4eec_49f4_bd7a_7f7e854f3485.slice/crio-e58093b70fa9e0bb046c10d2cb3dcace0cd738183b582fe2569f015b3d08982a WatchSource:0}: Error finding container e58093b70fa9e0bb046c10d2cb3dcace0cd738183b582fe2569f015b3d08982a: Status 404 returned error can't find the container with id e58093b70fa9e0bb046c10d2cb3dcace0cd738183b582fe2569f015b3d08982a
Nov 02 01:39:54.607779 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:54.607744    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt" event={"ID":"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485","Type":"ContainerStarted","Data":"e58093b70fa9e0bb046c10d2cb3dcace0cd738183b582fe2569f015b3d08982a"}
Nov 02 01:39:57.620321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:57.620281    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt" event={"ID":"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485","Type":"ContainerStarted","Data":"e0cdea67bdadbbed170f4ffa6677a04c02b62d2dd5d0b7c5e7610b9894390d8f"}
Nov 02 01:39:57.620708 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:57.620435    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:39:57.737036 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:39:57.736985    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt" podStartSLOduration=2.7711552900000003 podStartE2EDuration="5.736965929s" totalImagesPullingTime="2.965810639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:39:57.736337928 +0000 UTC m=+486.948707740" watchObservedRunningTime="2026-11-02 01:39:57.736965929 +0000 UTC m=+486.949335737"
Nov 02 01:40:03.629070 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:03.628983    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:40:04.614442 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:04.614404    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"]
Nov 02 01:40:04.617585 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:04.617567    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 01:40:04.647005 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:04.646977    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-openshift-service-ca\") pod \"maas-api-66b669b4fd-j675q\" (UID: \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\") " pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 01:40:04.647005 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:04.647005    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-metrics-certs\") pod \"maas-api-66b669b4fd-j675q\" (UID: \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\") " pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 01:40:04.647362 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:04.647032    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-maas-api-tls\") pod \"maas-api-66b669b4fd-j675q\" (UID: \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\") " pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 01:40:04.647362 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:04.647168    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5bs8\" (UniqueName: \"kubernetes.io/projected/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-kube-api-access-c5bs8\") pod \"maas-api-66b669b4fd-j675q\" (UID: \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\") " pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 01:40:04.677160 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:04.677108    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"]
Nov 02 01:40:04.747783 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:04.747753    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-maas-api-tls\") pod \"maas-api-66b669b4fd-j675q\" (UID: \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\") " pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 01:40:04.747929 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:04.747798    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5bs8\" (UniqueName: \"kubernetes.io/projected/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-kube-api-access-c5bs8\") pod \"maas-api-66b669b4fd-j675q\" (UID: \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\") " pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 01:40:04.747929 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:04.747839    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-openshift-service-ca\") pod \"maas-api-66b669b4fd-j675q\" (UID: \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\") " pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 01:40:04.747929 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:04.747855    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-metrics-certs\") pod \"maas-api-66b669b4fd-j675q\" (UID: \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\") " pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 01:40:04.748550 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:04.748522    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-openshift-service-ca\") pod \"maas-api-66b669b4fd-j675q\" (UID: \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\") " pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 01:40:04.750235 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:04.750216    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-maas-api-tls\") pod \"maas-api-66b669b4fd-j675q\" (UID: \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\") " pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 01:40:04.750326 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:04.750264    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-metrics-certs\") pod \"maas-api-66b669b4fd-j675q\" (UID: \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\") " pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 01:40:04.810428 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:04.810397    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5bs8\" (UniqueName: \"kubernetes.io/projected/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-kube-api-access-c5bs8\") pod \"maas-api-66b669b4fd-j675q\" (UID: \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\") " pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 01:40:04.929316 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:04.929237    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 01:40:05.298979 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:05.298942    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"]
Nov 02 01:40:05.305000 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:40:05.304968    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20040f1c_ca52_4c4c_af8f_fbe2ade458e4.slice/crio-e19f0098461dd48619f6a64dd5063556101cc245ed24ef0fbf5435b0e469d4fa WatchSource:0}: Error finding container e19f0098461dd48619f6a64dd5063556101cc245ed24ef0fbf5435b0e469d4fa: Status 404 returned error can't find the container with id e19f0098461dd48619f6a64dd5063556101cc245ed24ef0fbf5435b0e469d4fa
Nov 02 01:40:05.648380 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:05.648297    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q" event={"ID":"20040f1c-ca52-4c4c-af8f-fbe2ade458e4","Type":"ContainerStarted","Data":"e19f0098461dd48619f6a64dd5063556101cc245ed24ef0fbf5435b0e469d4fa"}
Nov 02 01:40:06.654023 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:06.653984    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q" event={"ID":"20040f1c-ca52-4c4c-af8f-fbe2ade458e4","Type":"ContainerStarted","Data":"4c892fddc2576f6e5fd94895fcc21962381da0903f38057f96a5882125f2543d"}
Nov 02 01:40:06.654442 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:06.654050    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 01:40:06.771359 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:06.771313    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q" podStartSLOduration=1.953288827 podStartE2EDuration="2.771297889s" totalImagesPullingTime="818.009062ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:40:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:40:06.770658927 +0000 UTC m=+495.983028749" watchObservedRunningTime="2026-11-02 01:40:06.771297889 +0000 UTC m=+495.983667732"
Nov 02 01:40:09.958152 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:40:09.958098    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:40:12.663286 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:12.663256    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 01:40:12.831468 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:12.831437    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"]
Nov 02 01:40:12.831697 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:12.831658    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt" podUID="d6e16e0a-4eec-49f4-bd7a-7f7e854f3485" containerName="maas-api" containerID="cri-o://e0cdea67bdadbbed170f4ffa6677a04c02b62d2dd5d0b7c5e7610b9894390d8f" gracePeriod=30
Nov 02 01:40:13.065317 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.065295    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:40:13.102413 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.102392    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-maas-api-tls\") pod \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\" (UID: \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\") "
Nov 02 01:40:13.102515 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.102431    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-metrics-certs\") pod \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\" (UID: \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\") "
Nov 02 01:40:13.102515 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.102474    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-kube-api-access-v7742\" (UniqueName: \"kubernetes.io/projected/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-kube-api-access-v7742\") pod \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\" (UID: \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\") "
Nov 02 01:40:13.102515 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.102502    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-openshift-service-ca\") pod \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\" (UID: \"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485\") "
Nov 02 01:40:13.102879 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.102856    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-openshift-service-ca" pod "d6e16e0a-4eec-49f4-bd7a-7f7e854f3485" (UID: "d6e16e0a-4eec-49f4-bd7a-7f7e854f3485"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:40:13.104544 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.104510    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-maas-api-tls" pod "d6e16e0a-4eec-49f4-bd7a-7f7e854f3485" (UID: "d6e16e0a-4eec-49f4-bd7a-7f7e854f3485"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 01:40:13.104634 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.104561    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-metrics-certs" pod "d6e16e0a-4eec-49f4-bd7a-7f7e854f3485" (UID: "d6e16e0a-4eec-49f4-bd7a-7f7e854f3485"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 01:40:13.104634 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.104588    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-kube-api-access-v7742" pod "d6e16e0a-4eec-49f4-bd7a-7f7e854f3485" (UID: "d6e16e0a-4eec-49f4-bd7a-7f7e854f3485"). InnerVolumeSpecName "kube-api-access-v7742". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 01:40:13.203944 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.203850    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-maas-api-tls\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:40:13.203944 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.203884    4515 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-metrics-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:40:13.203944 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.203898    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7742\" (UniqueName: \"kubernetes.io/projected/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-kube-api-access-v7742\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:40:13.203944 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.203909    4515 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485-openshift-service-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:40:13.680455 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.680370    4515 generic.go:356] "Generic (PLEG): container finished" podID="d6e16e0a-4eec-49f4-bd7a-7f7e854f3485" containerID="e0cdea67bdadbbed170f4ffa6677a04c02b62d2dd5d0b7c5e7610b9894390d8f" exitCode=0
Nov 02 01:40:13.680840 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.680449    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"
Nov 02 01:40:13.680840 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.680459    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt" event={"ID":"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485","Type":"ContainerDied","Data":"e0cdea67bdadbbed170f4ffa6677a04c02b62d2dd5d0b7c5e7610b9894390d8f"}
Nov 02 01:40:13.680840 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.680540    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt" event={"ID":"d6e16e0a-4eec-49f4-bd7a-7f7e854f3485","Type":"ContainerDied","Data":"e58093b70fa9e0bb046c10d2cb3dcace0cd738183b582fe2569f015b3d08982a"}
Nov 02 01:40:13.680840 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.680565    4515 scope.go:122] "RemoveContainer" containerID="e0cdea67bdadbbed170f4ffa6677a04c02b62d2dd5d0b7c5e7610b9894390d8f"
Nov 02 01:40:13.689060 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.689043    4515 scope.go:122] "RemoveContainer" containerID="e0cdea67bdadbbed170f4ffa6677a04c02b62d2dd5d0b7c5e7610b9894390d8f"
Nov 02 01:40:13.689363 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:40:13.689345    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0cdea67bdadbbed170f4ffa6677a04c02b62d2dd5d0b7c5e7610b9894390d8f\": container with ID starting with e0cdea67bdadbbed170f4ffa6677a04c02b62d2dd5d0b7c5e7610b9894390d8f not found: ID does not exist" containerID="e0cdea67bdadbbed170f4ffa6677a04c02b62d2dd5d0b7c5e7610b9894390d8f"
Nov 02 01:40:13.689430 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.689373    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0cdea67bdadbbed170f4ffa6677a04c02b62d2dd5d0b7c5e7610b9894390d8f"} err="failed to get container status \"e0cdea67bdadbbed170f4ffa6677a04c02b62d2dd5d0b7c5e7610b9894390d8f\": rpc error: code = NotFound desc = could not find container \"e0cdea67bdadbbed170f4ffa6677a04c02b62d2dd5d0b7c5e7610b9894390d8f\": container with ID starting with e0cdea67bdadbbed170f4ffa6677a04c02b62d2dd5d0b7c5e7610b9894390d8f not found: ID does not exist"
Nov 02 01:40:13.845173 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.845136    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"]
Nov 02 01:40:13.894699 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:13.894666    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-42pnt"]
Nov 02 01:40:14.951094 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:14.951059    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6e16e0a-4eec-49f4-bd7a-7f7e854f3485" path="/var/lib/kubelet/pods/d6e16e0a-4eec-49f4-bd7a-7f7e854f3485/volumes"
Nov 02 01:40:39.957184 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:40:39.957148    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:40:49.984570 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:49.984534    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-gnn75"]
Nov 02 01:40:49.984947 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:49.984784    4515 cpu_manager.go:414] "containerMap: removing container" podUID="d6e16e0a-4eec-49f4-bd7a-7f7e854f3485" containerName="maas-api"
Nov 02 01:40:49.984947 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:49.984793    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6e16e0a-4eec-49f4-bd7a-7f7e854f3485" containerName="maas-api"
Nov 02 01:40:49.984947 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:49.984849    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6e16e0a-4eec-49f4-bd7a-7f7e854f3485" containerName="maas-api"
Nov 02 01:40:49.987519 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:49.987503    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gnn75"
Nov 02 01:40:50.076595 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:50.076564    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h269h\" (UniqueName: \"kubernetes.io/projected/2cb9f581-50dc-4821-9ec3-3188bf13bc5a-kube-api-access-h269h\") pod \"llmisvc-controller-manager-6466c554d8-gnn75\" (UID: \"2cb9f581-50dc-4821-9ec3-3188bf13bc5a\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gnn75"
Nov 02 01:40:50.076927 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:50.076898    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cb9f581-50dc-4821-9ec3-3188bf13bc5a-cert\") pod \"llmisvc-controller-manager-6466c554d8-gnn75\" (UID: \"2cb9f581-50dc-4821-9ec3-3188bf13bc5a\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gnn75"
Nov 02 01:40:50.089220 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:50.089201    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Nov 02 01:40:50.089336 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:50.089293    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-jxj4b\""
Nov 02 01:40:50.090234 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:50.090215    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-gnn75"]
Nov 02 01:40:50.178061 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:50.178033    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h269h\" (UniqueName: \"kubernetes.io/projected/2cb9f581-50dc-4821-9ec3-3188bf13bc5a-kube-api-access-h269h\") pod \"llmisvc-controller-manager-6466c554d8-gnn75\" (UID: \"2cb9f581-50dc-4821-9ec3-3188bf13bc5a\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gnn75"
Nov 02 01:40:50.178218 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:50.178082    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cb9f581-50dc-4821-9ec3-3188bf13bc5a-cert\") pod \"llmisvc-controller-manager-6466c554d8-gnn75\" (UID: \"2cb9f581-50dc-4821-9ec3-3188bf13bc5a\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gnn75"
Nov 02 01:40:50.180513 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:50.180495    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cb9f581-50dc-4821-9ec3-3188bf13bc5a-cert\") pod \"llmisvc-controller-manager-6466c554d8-gnn75\" (UID: \"2cb9f581-50dc-4821-9ec3-3188bf13bc5a\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gnn75"
Nov 02 01:40:50.239420 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:50.239362    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h269h\" (UniqueName: \"kubernetes.io/projected/2cb9f581-50dc-4821-9ec3-3188bf13bc5a-kube-api-access-h269h\") pod \"llmisvc-controller-manager-6466c554d8-gnn75\" (UID: \"2cb9f581-50dc-4821-9ec3-3188bf13bc5a\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gnn75"
Nov 02 01:40:50.298967 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:50.298947    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gnn75"
Nov 02 01:40:50.667279 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:50.667203    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-gnn75"]
Nov 02 01:40:50.670734 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:40:50.670707    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cb9f581_50dc_4821_9ec3_3188bf13bc5a.slice/crio-d5a0b0e09599b955f6b1c57596fb857e17297f4b8acac99802e1b18478008209 WatchSource:0}: Error finding container d5a0b0e09599b955f6b1c57596fb857e17297f4b8acac99802e1b18478008209: Status 404 returned error can't find the container with id d5a0b0e09599b955f6b1c57596fb857e17297f4b8acac99802e1b18478008209
Nov 02 01:40:50.815675 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:50.815637    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gnn75" event={"ID":"2cb9f581-50dc-4821-9ec3-3188bf13bc5a","Type":"ContainerStarted","Data":"d5a0b0e09599b955f6b1c57596fb857e17297f4b8acac99802e1b18478008209"}
Nov 02 01:40:53.829711 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:53.829621    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gnn75" event={"ID":"2cb9f581-50dc-4821-9ec3-3188bf13bc5a","Type":"ContainerStarted","Data":"2a702326f93b0e36ac29c4762793b53df7559cb39eec3587264b521591b6d8e5"}
Nov 02 01:40:53.830083 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:53.829728    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gnn75"
Nov 02 01:40:53.945929 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:40:53.945862    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gnn75" podStartSLOduration=2.093844656 podStartE2EDuration="4.945847145s" totalImagesPullingTime="2.852002489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:40:53.945234832 +0000 UTC m=+543.157604641" watchObservedRunningTime="2026-11-02 01:40:53.945847145 +0000 UTC m=+543.158216952"
Nov 02 01:41:09.958342 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:41:09.958307    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:41:24.837913 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:24.837882    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gnn75"
Nov 02 01:41:39.957527 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:41:39.957491    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:41:46.365070 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.365037    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"]
Nov 02 01:41:46.367968 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.367951    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.414074 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.414034    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqvkd\" (UniqueName: \"kubernetes.io/projected/dab11213-84f0-4a27-bcd6-3f170312bba3-kube-api-access-xqvkd\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.414074 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.414071    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dab11213-84f0-4a27-bcd6-3f170312bba3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.414328 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.414089    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dab11213-84f0-4a27-bcd6-3f170312bba3-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.414328 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.414178    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dab11213-84f0-4a27-bcd6-3f170312bba3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.414328 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.414224    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dab11213-84f0-4a27-bcd6-3f170312bba3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.414328 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.414293    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dab11213-84f0-4a27-bcd6-3f170312bba3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.420587 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.420562    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 02 01:41:46.420587 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.420578    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 02 01:41:46.469958 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.469932    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Nov 02 01:41:46.470184 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.470167    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7vwh8\""
Nov 02 01:41:46.471243 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.471224    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"]
Nov 02 01:41:46.515096 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.515074    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dab11213-84f0-4a27-bcd6-3f170312bba3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.515219 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.515100    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dab11213-84f0-4a27-bcd6-3f170312bba3-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.515219 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.515138    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dab11213-84f0-4a27-bcd6-3f170312bba3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.515219 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.515187    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dab11213-84f0-4a27-bcd6-3f170312bba3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.515344 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.515244    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dab11213-84f0-4a27-bcd6-3f170312bba3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.515344 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.515299    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqvkd\" (UniqueName: \"kubernetes.io/projected/dab11213-84f0-4a27-bcd6-3f170312bba3-kube-api-access-xqvkd\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.515539 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.515510    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dab11213-84f0-4a27-bcd6-3f170312bba3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.515627 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.515554    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dab11213-84f0-4a27-bcd6-3f170312bba3-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.515699 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.515681    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dab11213-84f0-4a27-bcd6-3f170312bba3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.517387 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.517365    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dab11213-84f0-4a27-bcd6-3f170312bba3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.517528 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.517512    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dab11213-84f0-4a27-bcd6-3f170312bba3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.576244 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.576219    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqvkd\" (UniqueName: \"kubernetes.io/projected/dab11213-84f0-4a27-bcd6-3f170312bba3-kube-api-access-xqvkd\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m\" (UID: \"dab11213-84f0-4a27-bcd6-3f170312bba3\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:46.689027 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:46.688992    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:47.062495 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:47.062422    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"]
Nov 02 01:41:47.065469 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:41:47.065446    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddab11213_84f0_4a27_bcd6_3f170312bba3.slice/crio-368d88ec1a99b22f367aa3d2641bab771ad6aff7e97672a4195fce7cefe163c2 WatchSource:0}: Error finding container 368d88ec1a99b22f367aa3d2641bab771ad6aff7e97672a4195fce7cefe163c2: Status 404 returned error can't find the container with id 368d88ec1a99b22f367aa3d2641bab771ad6aff7e97672a4195fce7cefe163c2
Nov 02 01:41:48.029911 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:48.029873    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m" event={"ID":"dab11213-84f0-4a27-bcd6-3f170312bba3","Type":"ContainerStarted","Data":"368d88ec1a99b22f367aa3d2641bab771ad6aff7e97672a4195fce7cefe163c2"}
Nov 02 01:41:49.034988 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:49.034912    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m" event={"ID":"dab11213-84f0-4a27-bcd6-3f170312bba3","Type":"ContainerStarted","Data":"7cef43926d44ddb751977f526d1e0bef8bf89a98d1278fca960bd07db2e50523"}
Nov 02 01:41:49.152848 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:49.152795    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m" podStartSLOduration=1.5238264620000002 podStartE2EDuration="3.152780837s" totalImagesPullingTime="1.628954375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:41:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:41:49.151730737 +0000 UTC m=+598.364100545" watchObservedRunningTime="2026-11-02 01:41:49.152780837 +0000 UTC m=+598.365150646"
Nov 02 01:41:49.689919 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:49.689883    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:51.003254 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:51.003223    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 01:41:51.003686 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:51.003299    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 01:41:56.690230 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:56.690190    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:56.702773 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:56.702746    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:41:57.075307 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:41:57.075224    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:42:09.957863 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:42:09.957825    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:42:24.063089 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.063052    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"]
Nov 02 01:42:24.072659 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.072635    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.124889 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.124860    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-58w2j\""
Nov 02 01:42:24.124889 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.124867    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 02 01:42:24.204899 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.204867    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"]
Nov 02 01:42:24.216088 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.216057    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.216262 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.216101    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.216262 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.216152    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.216393 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.216273    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg45v\" (UniqueName: \"kubernetes.io/projected/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-kube-api-access-qg45v\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.216393 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.216309    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.216393 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.216388    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.216756 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.216409    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.216756 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.216432    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.216756 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.216525    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.317056 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.316986    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.317056 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.317027    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.317056 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.317051    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.317331 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.317086    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg45v\" (UniqueName: \"kubernetes.io/projected/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-kube-api-access-qg45v\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.317331 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.317133    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.317331 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.317182    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.317331 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.317205    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.317331 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.317236    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.317331 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.317266    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.317644 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.317516    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.317644 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.317565    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.317767 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.317712    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.317823 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.317762    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.318068 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.318042    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.319415 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.319391    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.319612 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.319592    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.377407 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.377381    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.377510 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.377462    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg45v\" (UniqueName: \"kubernetes.io/projected/dd24d725-bfe3-4cde-9fc7-5d6d4e497995-kube-api-access-qg45v\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7wmnn\" (UID: \"dd24d725-bfe3-4cde-9fc7-5d6d4e497995\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.385452 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.385430    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:24.809452 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:24.809421    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"]
Nov 02 01:42:24.813609 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:42:24.813578    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd24d725_bfe3_4cde_9fc7_5d6d4e497995.slice/crio-6d87e8c13ff8f5b1b53972d2893466098bde1d2e3826f8b19e8796e7daaf06df WatchSource:0}: Error finding container 6d87e8c13ff8f5b1b53972d2893466098bde1d2e3826f8b19e8796e7daaf06df: Status 404 returned error can't find the container with id 6d87e8c13ff8f5b1b53972d2893466098bde1d2e3826f8b19e8796e7daaf06df
Nov 02 01:42:25.168149 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:25.168050    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn" event={"ID":"dd24d725-bfe3-4cde-9fc7-5d6d4e497995","Type":"ContainerStarted","Data":"6d87e8c13ff8f5b1b53972d2893466098bde1d2e3826f8b19e8796e7daaf06df"}
Nov 02 01:42:28.786207 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:28.786173    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:42:28.786501 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:28.786268    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:42:28.786501 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:28.786307    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:42:29.188108 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:29.188059    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn" event={"ID":"dd24d725-bfe3-4cde-9fc7-5d6d4e497995","Type":"ContainerStarted","Data":"7b9396676e5151f0e34eab858d497385228f1778bc34137fc701d7d1fa25afc7"}
Nov 02 01:42:29.359246 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:29.359187    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn" podStartSLOduration=1.389536638 podStartE2EDuration="5.35917395s" totalImagesPullingTime="3.969637312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:42:29.358727209 +0000 UTC m=+638.571097020" watchObservedRunningTime="2026-11-02 01:42:29.35917395 +0000 UTC m=+638.571543757"
Nov 02 01:42:29.385714 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:29.385681    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:29.386871 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:29.386847    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-7wmnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Nov 02 01:42:29.386938 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:29.386910    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn" podUID="dd24d725-bfe3-4cde-9fc7-5d6d4e497995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Nov 02 01:42:30.386764 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:30.386728    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-7wmnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Nov 02 01:42:30.387151 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:30.386781    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn" podUID="dd24d725-bfe3-4cde-9fc7-5d6d4e497995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Nov 02 01:42:31.386137 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:31.386086    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-7wmnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Nov 02 01:42:31.386321 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:31.386174    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn" podUID="dd24d725-bfe3-4cde-9fc7-5d6d4e497995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Nov 02 01:42:32.386832 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:32.386799    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-7wmnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Nov 02 01:42:32.387248 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:32.386858    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn" podUID="dd24d725-bfe3-4cde-9fc7-5d6d4e497995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Nov 02 01:42:33.385815 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:33.385778    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-7wmnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Nov 02 01:42:33.385991 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:33.385840    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn" podUID="dd24d725-bfe3-4cde-9fc7-5d6d4e497995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Nov 02 01:42:34.386412 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:34.386378    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-7wmnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Nov 02 01:42:34.386802 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:34.386444    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn" podUID="dd24d725-bfe3-4cde-9fc7-5d6d4e497995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Nov 02 01:42:35.386270 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:35.386221    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-7wmnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Nov 02 01:42:35.386442 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:35.386292    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn" podUID="dd24d725-bfe3-4cde-9fc7-5d6d4e497995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Nov 02 01:42:36.926436 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:36.926400    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:36.926985 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:36.926916    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:36.927687 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:36.927670    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn"
Nov 02 01:42:39.198635 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.198602    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"]
Nov 02 01:42:39.201686 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.201670    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.240683 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.240646    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.240683 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.240675    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.240908 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.240697    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e6f4951-bf67-4011-ad79-e02a68472025-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.240908 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.240766    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e6f4951-bf67-4011-ad79-e02a68472025-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.240908 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.240854    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.241072 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.240954    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.241072 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.241008    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e6f4951-bf67-4011-ad79-e02a68472025-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.241072 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.241048    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp6dv\" (UniqueName: \"kubernetes.io/projected/1e6f4951-bf67-4011-ad79-e02a68472025-kube-api-access-gp6dv\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.241198 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.241147    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.296648 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.296602    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"]
Nov 02 01:42:39.341686 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.341660    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.341842 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.341690    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.341842 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.341712    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e6f4951-bf67-4011-ad79-e02a68472025-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.341842 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.341727    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e6f4951-bf67-4011-ad79-e02a68472025-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.341842 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.341749    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.341842 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.341782    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.341842 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.341827    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e6f4951-bf67-4011-ad79-e02a68472025-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.342219 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.341863    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp6dv\" (UniqueName: \"kubernetes.io/projected/1e6f4951-bf67-4011-ad79-e02a68472025-kube-api-access-gp6dv\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.342219 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.341904    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.342338 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.342283    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.342437 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.342417    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.342581 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.342561    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e6f4951-bf67-4011-ad79-e02a68472025-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.342764 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.342742    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.342910 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.342757    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.344306 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.344284    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.344492 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.344469    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e6f4951-bf67-4011-ad79-e02a68472025-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.406502 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.406473    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e6f4951-bf67-4011-ad79-e02a68472025-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.406654 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.406547    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp6dv\" (UniqueName: \"kubernetes.io/projected/1e6f4951-bf67-4011-ad79-e02a68472025-kube-api-access-gp6dv\") pod \"maas-default-gateway-openshift-default-778b7b5c47-n4f49\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.513702 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.513617    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:39.941492 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.941460    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"]
Nov 02 01:42:39.944761 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:42:39.944732    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e6f4951_bf67_4011_ad79_e02a68472025.slice/crio-579d86bb682df86aa9e7b9b93d30f9d36636e8e8f39b5b1c87e550ee452773d1 WatchSource:0}: Error finding container 579d86bb682df86aa9e7b9b93d30f9d36636e8e8f39b5b1c87e550ee452773d1: Status 404 returned error can't find the container with id 579d86bb682df86aa9e7b9b93d30f9d36636e8e8f39b5b1c87e550ee452773d1
Nov 02 01:42:39.948442 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.948410    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:42:39.948526 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.948482    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:42:39.948526 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:39.948512    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:42:39.962099 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:42:39.962070    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:42:40.234871 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:40.234786    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49" event={"ID":"1e6f4951-bf67-4011-ad79-e02a68472025","Type":"ContainerStarted","Data":"b1184a9e29db854e423d4ce2042ae468789b4875583eba32d89bdf3448002456"}
Nov 02 01:42:40.234871 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:40.234857    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49" event={"ID":"1e6f4951-bf67-4011-ad79-e02a68472025","Type":"ContainerStarted","Data":"579d86bb682df86aa9e7b9b93d30f9d36636e8e8f39b5b1c87e550ee452773d1"}
Nov 02 01:42:40.358412 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:40.358328    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49" podStartSLOduration=1.358308349 podStartE2EDuration="1.358308349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:42:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:42:40.357030919 +0000 UTC m=+649.569400728" watchObservedRunningTime="2026-11-02 01:42:40.358308349 +0000 UTC m=+649.570678200"
Nov 02 01:42:40.514512 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:40.514429    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:41.515970 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:41.515934    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-n4f49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 02 01:42:41.516413 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:41.515992    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49" podUID="1e6f4951-bf67-4011-ad79-e02a68472025" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 02 01:42:42.514931 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:42.514896    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-n4f49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 02 01:42:42.515095 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:42.514953    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49" podUID="1e6f4951-bf67-4011-ad79-e02a68472025" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 02 01:42:43.514793 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:43.514752    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-n4f49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 02 01:42:43.515208 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:43.514813    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49" podUID="1e6f4951-bf67-4011-ad79-e02a68472025" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 02 01:42:44.513987 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:44.513951    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-n4f49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 02 01:42:44.514178 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:44.514006    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49" podUID="1e6f4951-bf67-4011-ad79-e02a68472025" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 02 01:42:45.514781 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:45.514746    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-n4f49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 02 01:42:45.515182 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:45.514804    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49" podUID="1e6f4951-bf67-4011-ad79-e02a68472025" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 02 01:42:46.514288 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:46.514252    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-n4f49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 02 01:42:46.514456 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:46.514317    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49" podUID="1e6f4951-bf67-4011-ad79-e02a68472025" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 02 01:42:48.158299 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:48.158265    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:48.158769 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:48.158732    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:42:48.159430 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:42:48.159411    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:43:09.957665 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:43:09.957583    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:43:39.958328 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:43:39.958292    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:43:50.300271 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.300237    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"]
Nov 02 01:43:50.302391 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.302370    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.312958 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.312935    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-data\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.313048 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.312966    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgs2m\" (UniqueName: \"kubernetes.io/projected/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-kube-api-access-cgs2m\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.313088 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.313044    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-podinfo\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.313088 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.313071    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-token\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.313183 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.313106    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-credential-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.313232 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.313211    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-workload-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.313274 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.313242    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-envoy\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.313347 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.313320    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-workload-certs\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.313395 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.313383    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istiod-ca-cert\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.356826 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.356796    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-c62671-openshift-default-dockercfg-nw98n\""
Nov 02 01:43:50.407473 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.407433    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"]
Nov 02 01:43:50.414048 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.414022    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-workload-certs\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.414149 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.414064    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istiod-ca-cert\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.414249 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.414208    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-data\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.414308 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.414271    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgs2m\" (UniqueName: \"kubernetes.io/projected/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-kube-api-access-cgs2m\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.414360 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.414303    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-podinfo\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.414360 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.414336    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-token\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.414481 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.414459    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-workload-certs\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.414566 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.414547    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-credential-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.414631 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.414582    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-data\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.414631 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.414588    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-workload-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.414730 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.414651    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-envoy\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.414844 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.414821    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-credential-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.414934 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.414911    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istiod-ca-cert\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.415066 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.414959    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-workload-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.416758 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.416737    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-envoy\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.417224 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.417205    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-podinfo\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.475344 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.475315    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-token\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.475634 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.475610    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgs2m\" (UniqueName: \"kubernetes.io/projected/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-kube-api-access-cgs2m\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:50.613565 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:50.613480    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:51.050783 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:51.050747    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"]
Nov 02 01:43:51.053915 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:43:51.053888    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ee9e5c7_80ad_4744_9100_e7b9b5c50078.slice/crio-c35e018c60b10fe6f9e67ef77777bf1fe345c0535d47688e25da20d30d95bd03 WatchSource:0}: Error finding container c35e018c60b10fe6f9e67ef77777bf1fe345c0535d47688e25da20d30d95bd03: Status 404 returned error can't find the container with id c35e018c60b10fe6f9e67ef77777bf1fe345c0535d47688e25da20d30d95bd03
Nov 02 01:43:51.055932 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:51.055896    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:43:51.056024 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:51.055961    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:43:51.056024 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:51.055989    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:43:51.522066 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:51.522027    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl" event={"ID":"0ee9e5c7-80ad-4744-9100-e7b9b5c50078","Type":"ContainerStarted","Data":"2db996fe421cdea65bfd6940be99ab269c2e614470ae8ecd1a898899afaa4ff4"}
Nov 02 01:43:51.522473 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:51.522087    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl" event={"ID":"0ee9e5c7-80ad-4744-9100-e7b9b5c50078","Type":"ContainerStarted","Data":"c35e018c60b10fe6f9e67ef77777bf1fe345c0535d47688e25da20d30d95bd03"}
Nov 02 01:43:51.614166 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:51.614135    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:51.642976 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:51.642922    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl" podStartSLOduration=1.6429057889999998 podStartE2EDuration="1.642905789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:43:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:43:51.641557116 +0000 UTC m=+720.853926937" watchObservedRunningTime="2026-11-02 01:43:51.642905789 +0000 UTC m=+720.855275630"
Nov 02 01:43:52.619687 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:52.619658    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:53.529526 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:53.529490    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:43:53.530406 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:43:53.530386    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:44:04.312449 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:04.312409    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"]
Nov 02 01:44:04.314512 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:04.314496    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 01:44:04.366820 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:04.366793    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-9470a3-serving-cert\""
Nov 02 01:44:04.417844 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:04.417813    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"]
Nov 02 01:44:04.432057 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:04.432018    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de2ef1ec-9dc8-486e-949e-3d12cfdac897-maas-api-tls\") pod \"maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl\" (UID: \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 01:44:04.432232 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:04.432069    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxggv\" (UniqueName: \"kubernetes.io/projected/de2ef1ec-9dc8-486e-949e-3d12cfdac897-kube-api-access-dxggv\") pod \"maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl\" (UID: \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 01:44:04.432314 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:04.432270    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de2ef1ec-9dc8-486e-949e-3d12cfdac897-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl\" (UID: \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 01:44:04.432376 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:04.432328    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de2ef1ec-9dc8-486e-949e-3d12cfdac897-metrics-certs\") pod \"maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl\" (UID: \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 01:44:04.533405 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:04.533369    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de2ef1ec-9dc8-486e-949e-3d12cfdac897-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl\" (UID: \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 01:44:04.533405 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:04.533404    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de2ef1ec-9dc8-486e-949e-3d12cfdac897-metrics-certs\") pod \"maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl\" (UID: \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 01:44:04.533607 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:04.533454    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de2ef1ec-9dc8-486e-949e-3d12cfdac897-maas-api-tls\") pod \"maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl\" (UID: \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 01:44:04.533607 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:04.533476    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxggv\" (UniqueName: \"kubernetes.io/projected/de2ef1ec-9dc8-486e-949e-3d12cfdac897-kube-api-access-dxggv\") pod \"maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl\" (UID: \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 01:44:04.533607 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:44:04.533595    4515 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-serving-cert: secret "maas-api-e2e-worker-w0-9470a3-serving-cert" not found
Nov 02 01:44:04.533716 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:44:04.533673    4515 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de2ef1ec-9dc8-486e-949e-3d12cfdac897-maas-api-tls podName:de2ef1ec-9dc8-486e-949e-3d12cfdac897 nodeName:}" failed. No retries permitted until 2026-11-02 01:44:05.033654081 +0000 UTC m=+734.246023884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/de2ef1ec-9dc8-486e-949e-3d12cfdac897-maas-api-tls") pod "maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl" (UID: "de2ef1ec-9dc8-486e-949e-3d12cfdac897") : secret "maas-api-e2e-worker-w0-9470a3-serving-cert" not found
Nov 02 01:44:04.534058 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:04.534032    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de2ef1ec-9dc8-486e-949e-3d12cfdac897-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl\" (UID: \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 01:44:04.535815 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:04.535793    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de2ef1ec-9dc8-486e-949e-3d12cfdac897-metrics-certs\") pod \"maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl\" (UID: \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 01:44:04.595076 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:04.595005    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxggv\" (UniqueName: \"kubernetes.io/projected/de2ef1ec-9dc8-486e-949e-3d12cfdac897-kube-api-access-dxggv\") pod \"maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl\" (UID: \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 01:44:05.037653 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:05.037613    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de2ef1ec-9dc8-486e-949e-3d12cfdac897-maas-api-tls\") pod \"maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl\" (UID: \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 01:44:05.040012 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:05.039984    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de2ef1ec-9dc8-486e-949e-3d12cfdac897-maas-api-tls\") pod \"maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl\" (UID: \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 01:44:05.226298 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:05.226267    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 01:44:05.598585 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:05.598550    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"]
Nov 02 01:44:05.601453 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:44:05.601425    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde2ef1ec_9dc8_486e_949e_3d12cfdac897.slice/crio-cfd02f5b5330e2a4124c924cf74cc2940a4b02ac4a165b23c0c3294a7f6854bf WatchSource:0}: Error finding container cfd02f5b5330e2a4124c924cf74cc2940a4b02ac4a165b23c0c3294a7f6854bf: Status 404 returned error can't find the container with id cfd02f5b5330e2a4124c924cf74cc2940a4b02ac4a165b23c0c3294a7f6854bf
Nov 02 01:44:05.602871 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:05.602847    4515 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 01:44:06.580384 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:06.580343    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl" event={"ID":"de2ef1ec-9dc8-486e-949e-3d12cfdac897","Type":"ContainerStarted","Data":"baaa9e222edbc7014760a61f2ed5504b0756b7bae8d1a2f71dca40642843768d"}
Nov 02 01:44:06.580546 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:06.580404    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl" event={"ID":"de2ef1ec-9dc8-486e-949e-3d12cfdac897","Type":"ContainerStarted","Data":"cfd02f5b5330e2a4124c924cf74cc2940a4b02ac4a165b23c0c3294a7f6854bf"}
Nov 02 01:44:06.580546 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:06.580476    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 01:44:06.706976 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:06.706924    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl" podStartSLOduration=1.885388281 podStartE2EDuration="2.706909847s" totalImagesPullingTime="821.521566ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:44:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:44:06.705733687 +0000 UTC m=+735.918103497" watchObservedRunningTime="2026-11-02 01:44:06.706909847 +0000 UTC m=+735.919279656"
Nov 02 01:44:09.676025 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:09.675987    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"]
Nov 02 01:44:09.679335 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:09.679311    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 01:44:09.731493 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:09.731461    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-55bc84-serving-cert\""
Nov 02 01:44:09.781331 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:09.781297    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-maas-api-tls\") pod \"maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk\" (UID: \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 01:44:09.781483 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:09.781361    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-metrics-certs\") pod \"maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk\" (UID: \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 01:44:09.781528 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:09.781470    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk\" (UID: \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 01:44:09.781570 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:09.781557    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvjkh\" (UniqueName: \"kubernetes.io/projected/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-kube-api-access-pvjkh\") pod \"maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk\" (UID: \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 01:44:09.782761 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:09.782742    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"]
Nov 02 01:44:09.882251 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:09.882218    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-maas-api-tls\") pod \"maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk\" (UID: \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 01:44:09.882439 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:09.882263    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-metrics-certs\") pod \"maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk\" (UID: \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 01:44:09.882439 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:09.882294    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk\" (UID: \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 01:44:09.882439 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:09.882344    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvjkh\" (UniqueName: \"kubernetes.io/projected/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-kube-api-access-pvjkh\") pod \"maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk\" (UID: \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 01:44:09.882439 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:44:09.882405    4515 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-serving-cert: secret "maas-api-e2e-worker-w5-55bc84-serving-cert" not found
Nov 02 01:44:09.882590 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:44:09.882483    4515 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-maas-api-tls podName:b6442e7b-b2ad-404b-8f46-745b7c55c4ba nodeName:}" failed. No retries permitted until 2026-11-02 01:44:10.382461181 +0000 UTC m=+739.594830984 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-maas-api-tls") pod "maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk" (UID: "b6442e7b-b2ad-404b-8f46-745b7c55c4ba") : secret "maas-api-e2e-worker-w5-55bc84-serving-cert" not found
Nov 02 01:44:09.883077 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:09.883053    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk\" (UID: \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 01:44:09.884948 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:09.884925    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-metrics-certs\") pod \"maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk\" (UID: \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 01:44:09.943859 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:09.943827    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvjkh\" (UniqueName: \"kubernetes.io/projected/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-kube-api-access-pvjkh\") pod \"maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk\" (UID: \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 01:44:09.958926 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:44:09.958895    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:44:10.387631 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:10.387531    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-maas-api-tls\") pod \"maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk\" (UID: \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 01:44:10.390090 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:10.390064    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-maas-api-tls\") pod \"maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk\" (UID: \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 01:44:10.591549 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:10.591521    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 01:44:10.964921 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:10.964884    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"]
Nov 02 01:44:10.967887 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:44:10.967859    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6442e7b_b2ad_404b_8f46_745b7c55c4ba.slice/crio-21070ba3beeb924fa7cfb844fff4da4ac4f30d5bdc83a9d6d32a8a7ecaf3c204 WatchSource:0}: Error finding container 21070ba3beeb924fa7cfb844fff4da4ac4f30d5bdc83a9d6d32a8a7ecaf3c204: Status 404 returned error can't find the container with id 21070ba3beeb924fa7cfb844fff4da4ac4f30d5bdc83a9d6d32a8a7ecaf3c204
Nov 02 01:44:11.600621 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:11.600585    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk" event={"ID":"b6442e7b-b2ad-404b-8f46-745b7c55c4ba","Type":"ContainerStarted","Data":"21070ba3beeb924fa7cfb844fff4da4ac4f30d5bdc83a9d6d32a8a7ecaf3c204"}
Nov 02 01:44:12.589636 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:12.589608    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 01:44:12.606157 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:12.606112    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk" event={"ID":"b6442e7b-b2ad-404b-8f46-745b7c55c4ba","Type":"ContainerStarted","Data":"d8b9bff0e9bee21206b8d8bfb7a66051a9039c870a153592d1797db2add847d7"}
Nov 02 01:44:12.606335 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:12.606247    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 01:44:12.825076 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:12.825016    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk" podStartSLOduration=3.013994835 podStartE2EDuration="3.825001839s" totalImagesPullingTime="811.007004ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:44:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:44:12.823940102 +0000 UTC m=+742.036309912" watchObservedRunningTime="2026-11-02 01:44:12.825001839 +0000 UTC m=+742.037371646"
Nov 02 01:44:14.550669 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:14.550632    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"]
Nov 02 01:44:14.554014 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:14.553993    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 01:44:14.628345 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:14.628314    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3cb7579-408d-42e0-8ca2-9972bfb54b84-metrics-certs\") pod \"maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt\" (UID: \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 01:44:14.628345 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:14.628349    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwhsg\" (UniqueName: \"kubernetes.io/projected/b3cb7579-408d-42e0-8ca2-9972bfb54b84-kube-api-access-lwhsg\") pod \"maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt\" (UID: \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 01:44:14.628564 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:14.628461    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3cb7579-408d-42e0-8ca2-9972bfb54b84-maas-api-tls\") pod \"maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt\" (UID: \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 01:44:14.628564 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:14.628535    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3cb7579-408d-42e0-8ca2-9972bfb54b84-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt\" (UID: \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 01:44:14.635753 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:14.635728    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-dbe849-serving-cert\""
Nov 02 01:44:14.686996 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:14.686955    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"]
Nov 02 01:44:14.729344 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:14.729316    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwhsg\" (UniqueName: \"kubernetes.io/projected/b3cb7579-408d-42e0-8ca2-9972bfb54b84-kube-api-access-lwhsg\") pod \"maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt\" (UID: \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 01:44:14.729516 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:14.729374    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3cb7579-408d-42e0-8ca2-9972bfb54b84-maas-api-tls\") pod \"maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt\" (UID: \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 01:44:14.729516 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:14.729425    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3cb7579-408d-42e0-8ca2-9972bfb54b84-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt\" (UID: \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 01:44:14.729516 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:14.729474    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3cb7579-408d-42e0-8ca2-9972bfb54b84-metrics-certs\") pod \"maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt\" (UID: \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 01:44:14.729516 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:44:14.729506    4515 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-serving-cert: secret "maas-api-e2e-worker-w1-dbe849-serving-cert" not found
Nov 02 01:44:14.729676 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:44:14.729588    4515 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3cb7579-408d-42e0-8ca2-9972bfb54b84-maas-api-tls podName:b3cb7579-408d-42e0-8ca2-9972bfb54b84 nodeName:}" failed. No retries permitted until 2026-11-02 01:44:15.229565105 +0000 UTC m=+744.441934917 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b3cb7579-408d-42e0-8ca2-9972bfb54b84-maas-api-tls") pod "maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt" (UID: "b3cb7579-408d-42e0-8ca2-9972bfb54b84") : secret "maas-api-e2e-worker-w1-dbe849-serving-cert" not found
Nov 02 01:44:14.730229 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:14.730209    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3cb7579-408d-42e0-8ca2-9972bfb54b84-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt\" (UID: \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 01:44:14.731872 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:14.731851    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3cb7579-408d-42e0-8ca2-9972bfb54b84-metrics-certs\") pod \"maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt\" (UID: \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 01:44:14.793057 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:14.793026    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwhsg\" (UniqueName: \"kubernetes.io/projected/b3cb7579-408d-42e0-8ca2-9972bfb54b84-kube-api-access-lwhsg\") pod \"maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt\" (UID: \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 01:44:15.235322 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:15.235278    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3cb7579-408d-42e0-8ca2-9972bfb54b84-maas-api-tls\") pod \"maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt\" (UID: \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 01:44:15.237775 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:15.237742    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3cb7579-408d-42e0-8ca2-9972bfb54b84-maas-api-tls\") pod \"maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt\" (UID: \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 01:44:15.466618 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:15.466581    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 01:44:15.840776 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:15.840698    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"]
Nov 02 01:44:15.843902 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:44:15.843864    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3cb7579_408d_42e0_8ca2_9972bfb54b84.slice/crio-88e851f476d71ec9c37918ab369fdcb01d6b2996ba3cde476ca76883cf074024 WatchSource:0}: Error finding container 88e851f476d71ec9c37918ab369fdcb01d6b2996ba3cde476ca76883cf074024: Status 404 returned error can't find the container with id 88e851f476d71ec9c37918ab369fdcb01d6b2996ba3cde476ca76883cf074024
Nov 02 01:44:16.625305 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:16.625270    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt" event={"ID":"b3cb7579-408d-42e0-8ca2-9972bfb54b84","Type":"ContainerStarted","Data":"88e851f476d71ec9c37918ab369fdcb01d6b2996ba3cde476ca76883cf074024"}
Nov 02 01:44:17.631005 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:17.630971    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt" event={"ID":"b3cb7579-408d-42e0-8ca2-9972bfb54b84","Type":"ContainerStarted","Data":"4f1bc9b7618c8c5f0d8c9761cb179cf452eb9cb5b3545ada7ea6de3fcedc46b2"}
Nov 02 01:44:17.631500 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:17.631106    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 01:44:17.755882 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:17.755781    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt" podStartSLOduration=2.965585632 podStartE2EDuration="3.755765615s" totalImagesPullingTime="790.179983ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:44:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:44:17.755180962 +0000 UTC m=+746.967550778" watchObservedRunningTime="2026-11-02 01:44:17.755765615 +0000 UTC m=+746.968135423"
Nov 02 01:44:18.615435 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:18.615406    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 01:44:23.641155 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:23.641110    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 01:44:29.184784 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:29.184742    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"]
Nov 02 01:44:29.189502 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:29.189479    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 01:44:29.241382 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:29.241345    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-0ebdb3-serving-cert\""
Nov 02 01:44:29.292422 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:29.292387    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"]
Nov 02 01:44:29.364774 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:29.364739    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a387928-4b07-4d1b-b906-89d36cfe2f0e-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw\" (UID: \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 01:44:29.364774 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:29.364775    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a387928-4b07-4d1b-b906-89d36cfe2f0e-metrics-certs\") pod \"maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw\" (UID: \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 01:44:29.364994 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:29.364805    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn46p\" (UniqueName: \"kubernetes.io/projected/1a387928-4b07-4d1b-b906-89d36cfe2f0e-kube-api-access-vn46p\") pod \"maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw\" (UID: \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 01:44:29.364994 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:29.364975    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a387928-4b07-4d1b-b906-89d36cfe2f0e-maas-api-tls\") pod \"maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw\" (UID: \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 01:44:29.465671 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:29.465633    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a387928-4b07-4d1b-b906-89d36cfe2f0e-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw\" (UID: \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 01:44:29.465671 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:29.465671    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a387928-4b07-4d1b-b906-89d36cfe2f0e-metrics-certs\") pod \"maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw\" (UID: \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 01:44:29.465905 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:29.465696    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn46p\" (UniqueName: \"kubernetes.io/projected/1a387928-4b07-4d1b-b906-89d36cfe2f0e-kube-api-access-vn46p\") pod \"maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw\" (UID: \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 01:44:29.465905 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:29.465720    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a387928-4b07-4d1b-b906-89d36cfe2f0e-maas-api-tls\") pod \"maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw\" (UID: \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 01:44:29.465905 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:44:29.465806    4515 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-serving-cert: secret "maas-api-e2e-worker-w6-0ebdb3-serving-cert" not found
Nov 02 01:44:29.465905 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:44:29.465868    4515 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a387928-4b07-4d1b-b906-89d36cfe2f0e-maas-api-tls podName:1a387928-4b07-4d1b-b906-89d36cfe2f0e nodeName:}" failed. No retries permitted until 2026-11-02 01:44:29.965853778 +0000 UTC m=+759.178223563 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1a387928-4b07-4d1b-b906-89d36cfe2f0e-maas-api-tls") pod "maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw" (UID: "1a387928-4b07-4d1b-b906-89d36cfe2f0e") : secret "maas-api-e2e-worker-w6-0ebdb3-serving-cert" not found
Nov 02 01:44:29.466430 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:29.466412    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a387928-4b07-4d1b-b906-89d36cfe2f0e-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw\" (UID: \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 01:44:29.468103 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:29.468084    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a387928-4b07-4d1b-b906-89d36cfe2f0e-metrics-certs\") pod \"maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw\" (UID: \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 01:44:29.529474 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:29.529440    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn46p\" (UniqueName: \"kubernetes.io/projected/1a387928-4b07-4d1b-b906-89d36cfe2f0e-kube-api-access-vn46p\") pod \"maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw\" (UID: \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 01:44:29.969708 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:29.969671    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a387928-4b07-4d1b-b906-89d36cfe2f0e-maas-api-tls\") pod \"maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw\" (UID: \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 01:44:29.972059 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:29.972031    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a387928-4b07-4d1b-b906-89d36cfe2f0e-maas-api-tls\") pod \"maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw\" (UID: \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 01:44:30.102061 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:30.102020    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 01:44:30.473685 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:30.473658    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"]
Nov 02 01:44:30.476790 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:44:30.476762    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a387928_4b07_4d1b_b906_89d36cfe2f0e.slice/crio-173ae12144019a2f1907637eaad49ad61af1aed1a7ffd99f8bca821a1e9e2e6c WatchSource:0}: Error finding container 173ae12144019a2f1907637eaad49ad61af1aed1a7ffd99f8bca821a1e9e2e6c: Status 404 returned error can't find the container with id 173ae12144019a2f1907637eaad49ad61af1aed1a7ffd99f8bca821a1e9e2e6c
Nov 02 01:44:30.683060 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:30.683026    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw" event={"ID":"1a387928-4b07-4d1b-b906-89d36cfe2f0e","Type":"ContainerStarted","Data":"173ae12144019a2f1907637eaad49ad61af1aed1a7ffd99f8bca821a1e9e2e6c"}
Nov 02 01:44:31.688335 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:31.688305    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw" event={"ID":"1a387928-4b07-4d1b-b906-89d36cfe2f0e","Type":"ContainerStarted","Data":"83bd5d3d579e3a7a73ab23bad2120e87b75c1bc9a54c259efdc9e198d79beb9e"}
Nov 02 01:44:31.688718 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:31.688376    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 01:44:31.805499 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:31.805445    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw" podStartSLOduration=2.009880587 podStartE2EDuration="2.805430025s" totalImagesPullingTime="795.549438ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:44:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:44:31.804604947 +0000 UTC m=+761.016974758" watchObservedRunningTime="2026-11-02 01:44:31.805430025 +0000 UTC m=+761.017799833"
Nov 02 01:44:37.698228 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:37.698161    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 01:44:39.958169 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:44:39.958135    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:44:43.922062 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:43.922030    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"]
Nov 02 01:44:43.925328 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:43.925304    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 01:44:43.977473 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:43.977437    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-c62671-serving-cert\""
Nov 02 01:44:43.996106 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:43.996072    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c81aedb-70f7-4846-b97b-ead08257737a-maas-api-tls\") pod \"maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4\" (UID: \"8c81aedb-70f7-4846-b97b-ead08257737a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 01:44:43.996277 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:43.996138    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcr6x\" (UniqueName: \"kubernetes.io/projected/8c81aedb-70f7-4846-b97b-ead08257737a-kube-api-access-lcr6x\") pod \"maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4\" (UID: \"8c81aedb-70f7-4846-b97b-ead08257737a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 01:44:43.996277 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:43.996239    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c81aedb-70f7-4846-b97b-ead08257737a-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4\" (UID: \"8c81aedb-70f7-4846-b97b-ead08257737a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 01:44:43.996412 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:43.996390    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c81aedb-70f7-4846-b97b-ead08257737a-metrics-certs\") pod \"maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4\" (UID: \"8c81aedb-70f7-4846-b97b-ead08257737a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 01:44:44.028549 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:44.028510    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"]
Nov 02 01:44:44.096856 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:44.096819    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c81aedb-70f7-4846-b97b-ead08257737a-metrics-certs\") pod \"maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4\" (UID: \"8c81aedb-70f7-4846-b97b-ead08257737a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 01:44:44.097041 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:44.096867    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c81aedb-70f7-4846-b97b-ead08257737a-maas-api-tls\") pod \"maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4\" (UID: \"8c81aedb-70f7-4846-b97b-ead08257737a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 01:44:44.097041 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:44.096898    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcr6x\" (UniqueName: \"kubernetes.io/projected/8c81aedb-70f7-4846-b97b-ead08257737a-kube-api-access-lcr6x\") pod \"maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4\" (UID: \"8c81aedb-70f7-4846-b97b-ead08257737a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 01:44:44.097041 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:44.096935    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c81aedb-70f7-4846-b97b-ead08257737a-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4\" (UID: \"8c81aedb-70f7-4846-b97b-ead08257737a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 01:44:44.097251 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:44:44.097164    4515 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-serving-cert: secret "maas-api-e2e-worker-w2-c62671-serving-cert" not found
Nov 02 01:44:44.097251 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:44:44.097250    4515 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c81aedb-70f7-4846-b97b-ead08257737a-maas-api-tls podName:8c81aedb-70f7-4846-b97b-ead08257737a nodeName:}" failed. No retries permitted until 2026-11-02 01:44:44.597220621 +0000 UTC m=+773.809590408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8c81aedb-70f7-4846-b97b-ead08257737a-maas-api-tls") pod "maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4" (UID: "8c81aedb-70f7-4846-b97b-ead08257737a") : secret "maas-api-e2e-worker-w2-c62671-serving-cert" not found
Nov 02 01:44:44.097911 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:44.097877    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c81aedb-70f7-4846-b97b-ead08257737a-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4\" (UID: \"8c81aedb-70f7-4846-b97b-ead08257737a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 01:44:44.099499 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:44.099477    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c81aedb-70f7-4846-b97b-ead08257737a-metrics-certs\") pod \"maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4\" (UID: \"8c81aedb-70f7-4846-b97b-ead08257737a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 01:44:44.159990 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:44.159953    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcr6x\" (UniqueName: \"kubernetes.io/projected/8c81aedb-70f7-4846-b97b-ead08257737a-kube-api-access-lcr6x\") pod \"maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4\" (UID: \"8c81aedb-70f7-4846-b97b-ead08257737a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 01:44:44.600970 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:44.600933    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c81aedb-70f7-4846-b97b-ead08257737a-maas-api-tls\") pod \"maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4\" (UID: \"8c81aedb-70f7-4846-b97b-ead08257737a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 01:44:44.603369 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:44.603343    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c81aedb-70f7-4846-b97b-ead08257737a-maas-api-tls\") pod \"maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4\" (UID: \"8c81aedb-70f7-4846-b97b-ead08257737a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 01:44:44.837413 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:44.837380    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 01:44:45.207333 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:45.207307    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"]
Nov 02 01:44:45.210701 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:44:45.210673    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c81aedb_70f7_4846_b97b_ead08257737a.slice/crio-bd1c1d57f940cc07dbb97487080f552c4bc0a0bf3e396030a73bd8f60d0b31d4 WatchSource:0}: Error finding container bd1c1d57f940cc07dbb97487080f552c4bc0a0bf3e396030a73bd8f60d0b31d4: Status 404 returned error can't find the container with id bd1c1d57f940cc07dbb97487080f552c4bc0a0bf3e396030a73bd8f60d0b31d4
Nov 02 01:44:45.746403 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:45.746366    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4" event={"ID":"8c81aedb-70f7-4846-b97b-ead08257737a","Type":"ContainerStarted","Data":"bd1c1d57f940cc07dbb97487080f552c4bc0a0bf3e396030a73bd8f60d0b31d4"}
Nov 02 01:44:46.751805 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:46.751770    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4" event={"ID":"8c81aedb-70f7-4846-b97b-ead08257737a","Type":"ContainerStarted","Data":"c43799e8ad388c34c693b35e6d5bf000f2b449de5cf86b0ee9b9936f8b8404f1"}
Nov 02 01:44:46.752222 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:46.751848    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 01:44:46.869822 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:46.869766    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4" podStartSLOduration=3.051815229 podStartE2EDuration="3.86975226s" totalImagesPullingTime="817.937031ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:44:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:44:46.868777652 +0000 UTC m=+776.081147458" watchObservedRunningTime="2026-11-02 01:44:46.86975226 +0000 UTC m=+776.082122067"
Nov 02 01:44:52.762288 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:52.762259    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 01:44:53.338768 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.338732    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"]
Nov 02 01:44:53.349454 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.349414    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.379930 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.379897    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/228f299b-731d-4375-93a9-fa777801b0a1-tls-certs\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.380156 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.379946    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-model-cache\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.380156 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.380053    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-dshm\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.380156 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.380104    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-tmp-dir\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.380340 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.380215    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-home\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.380340 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.380237    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7zfl\" (UniqueName: \"kubernetes.io/projected/228f299b-731d-4375-93a9-fa777801b0a1-kube-api-access-h7zfl\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.398597 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.398563    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"]
Nov 02 01:44:53.449440 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.449397    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-9470a3\"/\"kube-root-ca.crt\""
Nov 02 01:44:53.449718 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.449429    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-9470a3\"/\"default-dockercfg-mcj8f\""
Nov 02 01:44:53.449718 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.449438    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-9470a3\"/\"simulator-w0-9470a3-kserve-self-signed-certs\""
Nov 02 01:44:53.449895 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.449490    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-9470a3\"/\"openshift-service-ca.crt\""
Nov 02 01:44:53.481383 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.481345    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-model-cache\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.481583 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.481425    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-dshm\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.481583 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.481454    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-tmp-dir\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.481583 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.481529    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-home\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.481583 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.481553    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7zfl\" (UniqueName: \"kubernetes.io/projected/228f299b-731d-4375-93a9-fa777801b0a1-kube-api-access-h7zfl\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.481818 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.481596    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/228f299b-731d-4375-93a9-fa777801b0a1-tls-certs\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.482220 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.482075    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-model-cache\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.482220 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.482102    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-home\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.482437 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.482417    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-tmp-dir\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.484172 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.484148    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-dshm\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.484841 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.484808    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/228f299b-731d-4375-93a9-fa777801b0a1-tls-certs\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.551728 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.551690    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7zfl\" (UniqueName: \"kubernetes.io/projected/228f299b-731d-4375-93a9-fa777801b0a1-kube-api-access-h7zfl\") pod \"simulator-w0-9470a3-kserve-dd5f6b769-r2tqv\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") " pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:53.662582 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:53.662489    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:44:54.045143 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:54.040983    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"]
Nov 02 01:44:54.052724 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:44:54.052695    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod228f299b_731d_4375_93a9_fa777801b0a1.slice/crio-3783c3f13448b504092fea9e2e4c228c542bb76e369c4042faa7b8d4baf658c4 WatchSource:0}: Error finding container 3783c3f13448b504092fea9e2e4c228c542bb76e369c4042faa7b8d4baf658c4: Status 404 returned error can't find the container with id 3783c3f13448b504092fea9e2e4c228c542bb76e369c4042faa7b8d4baf658c4
Nov 02 01:44:54.786937 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:54.786904    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" event={"ID":"228f299b-731d-4375-93a9-fa777801b0a1","Type":"ContainerStarted","Data":"245e32b57808dee6131e0e7217b67c3cb0f61e0fbbd02a39d821492eb0a224ed"}
Nov 02 01:44:54.787143 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:54.786948    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" event={"ID":"228f299b-731d-4375-93a9-fa777801b0a1","Type":"ContainerStarted","Data":"3783c3f13448b504092fea9e2e4c228c542bb76e369c4042faa7b8d4baf658c4"}
Nov 02 01:44:54.911690 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:54.911632    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" podStartSLOduration=1.91161747 podStartE2EDuration="1.91161747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:44:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:44:54.910142429 +0000 UTC m=+784.122512238" watchObservedRunningTime="2026-11-02 01:44:54.91161747 +0000 UTC m=+784.123987278"
Nov 02 01:44:55.662881 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:44:55.662846    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:45:02.821495 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:02.821460    4515 generic.go:356] "Generic (PLEG): container finished" podID="0ee9e5c7-80ad-4744-9100-e7b9b5c50078" containerID="2db996fe421cdea65bfd6940be99ab269c2e614470ae8ecd1a898899afaa4ff4" exitCode=0
Nov 02 01:45:02.821948 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:02.821530    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl" event={"ID":"0ee9e5c7-80ad-4744-9100-e7b9b5c50078","Type":"ContainerDied","Data":"2db996fe421cdea65bfd6940be99ab269c2e614470ae8ecd1a898899afaa4ff4"}
Nov 02 01:45:02.821997 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:02.821951    4515 scope.go:122] "RemoveContainer" containerID="2db996fe421cdea65bfd6940be99ab269c2e614470ae8ecd1a898899afaa4ff4"
Nov 02 01:45:02.823243 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:02.823211    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:45:02.823351 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:02.823288    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:45:02.823351 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:02.823338    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:45:03.663019 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:03.662987    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:45:03.675715 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:03.675692    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:45:03.849826 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:03.849789    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl" event={"ID":"0ee9e5c7-80ad-4744-9100-e7b9b5c50078","Type":"ContainerStarted","Data":"fff9efa4ad4658e344fa3df144b82394446dd4b6b6bbd9552430bc152a888b3d"}
Nov 02 01:45:03.861394 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:03.861373    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:45:04.614582 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:04.614544    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:45:04.615539 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:04.615514    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 02 01:45:04.615651 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:04.615563    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl" podUID="0ee9e5c7-80ad-4744-9100-e7b9b5c50078" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 02 01:45:05.613818 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:05.613782    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 02 01:45:05.614224 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:05.613851    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl" podUID="0ee9e5c7-80ad-4744-9100-e7b9b5c50078" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 02 01:45:06.614216 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:06.614179    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 02 01:45:06.614615 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:06.614259    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl" podUID="0ee9e5c7-80ad-4744-9100-e7b9b5c50078" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 02 01:45:07.614287 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:07.614251    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 02 01:45:07.614691 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:07.614323    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl" podUID="0ee9e5c7-80ad-4744-9100-e7b9b5c50078" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 02 01:45:08.542141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:08.533216    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:45:08.614244 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:08.614201    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 02 01:45:08.614425 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:08.614265    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl" podUID="0ee9e5c7-80ad-4744-9100-e7b9b5c50078" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 02 01:45:09.617978 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:09.617943    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:45:09.619035 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:09.619015    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 01:45:09.963361 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:45:09.963322    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:45:18.810274 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.810242    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"]
Nov 02 01:45:18.813640 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.813619    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:18.861566 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.861534    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f21efa8-1bf5-449f-addd-10e78c8732dc-tls-certs\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:18.861566 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.861579    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-home\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:18.861794 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.861610    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-dshm\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:18.861794 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.861692    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-tmp-dir\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:18.861794 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.861742    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp4lj\" (UniqueName: \"kubernetes.io/projected/6f21efa8-1bf5-449f-addd-10e78c8732dc-kube-api-access-fp4lj\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:18.861794 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.861769    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-model-cache\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:18.915309 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.915272    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-55bc84\"/\"openshift-service-ca.crt\""
Nov 02 01:45:18.915475 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.915448    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-55bc84\"/\"simulator-w5-55bc84-kserve-self-signed-certs\""
Nov 02 01:45:18.915564 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.915520    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-55bc84\"/\"default-dockercfg-m62zr\""
Nov 02 01:45:18.915601 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.915580    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-55bc84\"/\"kube-root-ca.crt\""
Nov 02 01:45:18.916964 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.916944    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"]
Nov 02 01:45:18.962794 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.962762    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp4lj\" (UniqueName: \"kubernetes.io/projected/6f21efa8-1bf5-449f-addd-10e78c8732dc-kube-api-access-fp4lj\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:18.962992 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.962802    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-model-cache\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:18.962992 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.962845    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f21efa8-1bf5-449f-addd-10e78c8732dc-tls-certs\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:18.962992 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.962879    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-home\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:18.962992 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.962900    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-dshm\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:18.962992 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.962954    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-tmp-dir\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:18.963353 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.963332    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-home\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:18.963418 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.963362    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-tmp-dir\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:18.963590 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.963572    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-model-cache\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:18.965296 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.965271    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-dshm\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:18.965667 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:18.965648    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f21efa8-1bf5-449f-addd-10e78c8732dc-tls-certs\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:19.023873 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:19.023845    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp4lj\" (UniqueName: \"kubernetes.io/projected/6f21efa8-1bf5-449f-addd-10e78c8732dc-kube-api-access-fp4lj\") pod \"simulator-w5-55bc84-kserve-69cd684c9f-dqgzt\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") " pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:19.125428 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:19.125325    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:19.507130 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:19.507077    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"]
Nov 02 01:45:19.509902 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:45:19.509873    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f21efa8_1bf5_449f_addd_10e78c8732dc.slice/crio-d5186d5680c80712be254ee813236e549e8b67ca6706cd1f80d3736b7e84a8d3 WatchSource:0}: Error finding container d5186d5680c80712be254ee813236e549e8b67ca6706cd1f80d3736b7e84a8d3: Status 404 returned error can't find the container with id d5186d5680c80712be254ee813236e549e8b67ca6706cd1f80d3736b7e84a8d3
Nov 02 01:45:19.925706 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:19.925608    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" event={"ID":"6f21efa8-1bf5-449f-addd-10e78c8732dc","Type":"ContainerStarted","Data":"f3c7e82b076b05d0d351f2baef21136876a146fc4529be88af9b9bfcf5c295bc"}
Nov 02 01:45:19.925706 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:19.925651    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" event={"ID":"6f21efa8-1bf5-449f-addd-10e78c8732dc","Type":"ContainerStarted","Data":"d5186d5680c80712be254ee813236e549e8b67ca6706cd1f80d3736b7e84a8d3"}
Nov 02 01:45:20.042293 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:20.042238    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" podStartSLOduration=2.042223786 podStartE2EDuration="2.042223786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:45:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:45:20.040980115 +0000 UTC m=+809.253349924" watchObservedRunningTime="2026-11-02 01:45:20.042223786 +0000 UTC m=+809.254593594"
Nov 02 01:45:20.125459 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:20.125415    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:26.575509 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.575471    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"]
Nov 02 01:45:26.579149 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.579134    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.628368 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.628337    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-home\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.628368 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.628370    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-dshm\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.628545 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.628386    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-tmp-dir\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.628545 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.628407    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-tls-certs\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.628545 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.628506    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-model-cache\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.628651 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.628548    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltmg5\" (UniqueName: \"kubernetes.io/projected/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-kube-api-access-ltmg5\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.630907 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.630882    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-dbe849\"/\"openshift-service-ca.crt\""
Nov 02 01:45:26.682056 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.682028    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"]
Nov 02 01:45:26.715853 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.715829    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-dbe849\"/\"default-dockercfg-wkvbk\""
Nov 02 01:45:26.715853 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.715832    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-dbe849\"/\"simulator-w1-dbe849-kserve-self-signed-certs\""
Nov 02 01:45:26.729390 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.729357    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-home\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.729545 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.729405    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-dshm\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.729609 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.729553    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-tmp-dir\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.729670 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.729634    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-tls-certs\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.729722 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.729682    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-model-cache\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.729769 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.729752    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltmg5\" (UniqueName: \"kubernetes.io/projected/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-kube-api-access-ltmg5\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.730176 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.730109    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-tmp-dir\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.730176 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.730149    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-model-cache\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.730176 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.730202    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-home\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.730176 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.731083    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-dbe849\"/\"kube-root-ca.crt\""
Nov 02 01:45:26.732331 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.732313    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-dshm\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.732942 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.732920    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-tls-certs\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.790783 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.790753    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltmg5\" (UniqueName: \"kubernetes.io/projected/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-kube-api-access-ltmg5\") pod \"simulator-w1-dbe849-kserve-698d9cf47b-cxhqp\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") " pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:26.890890 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:26.890807    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:27.276075 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:27.276051    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"]
Nov 02 01:45:27.278623 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:45:27.278594    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f78c3ae_3bc7_49a5_ad5c_39b6c02d4fbc.slice/crio-bc34c74efafd3a407c3108cac220e5dfeb4e33b3f238224992308944c2bf208d WatchSource:0}: Error finding container bc34c74efafd3a407c3108cac220e5dfeb4e33b3f238224992308944c2bf208d: Status 404 returned error can't find the container with id bc34c74efafd3a407c3108cac220e5dfeb4e33b3f238224992308944c2bf208d
Nov 02 01:45:27.969252 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:27.969221    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" event={"ID":"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc","Type":"ContainerStarted","Data":"8b62a7277f60540b0cb20b025f942f2e087c792b1dded97b5ab3dab144408867"}
Nov 02 01:45:27.969647 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:27.969264    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" event={"ID":"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc","Type":"ContainerStarted","Data":"bc34c74efafd3a407c3108cac220e5dfeb4e33b3f238224992308944c2bf208d"}
Nov 02 01:45:28.085872 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:28.085822    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" podStartSLOduration=2.085808646 podStartE2EDuration="2.085808646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:45:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:45:28.084816155 +0000 UTC m=+817.297185962" watchObservedRunningTime="2026-11-02 01:45:28.085808646 +0000 UTC m=+817.298178455"
Nov 02 01:45:28.891362 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:28.891317    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:29.125807 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:29.125777    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:29.138393 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:29.138361    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:29.988253 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:29.988221    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:45:36.687762 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.687727    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"]
Nov 02 01:45:36.691615 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.691591    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.793405 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.793376    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-ed0600\"/\"openshift-service-ca.crt\""
Nov 02 01:45:36.793405 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.793376    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-ed0600\"/\"kube-root-ca.crt\""
Nov 02 01:45:36.793625 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.793525    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-ed0600\"/\"simulator-w4-ed0600-kserve-self-signed-certs\""
Nov 02 01:45:36.793667 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.793621    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-ed0600\"/\"default-dockercfg-qr9gk\""
Nov 02 01:45:36.795160 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.795139    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"]
Nov 02 01:45:36.820731 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.820704    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-tmp-dir\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.820845 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.820771    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-home\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.820888 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.820845    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgbxm\" (UniqueName: \"kubernetes.io/projected/747c597f-77cf-4287-a7a1-879565aec8ae-kube-api-access-lgbxm\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.820888 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.820868    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-dshm\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.820888 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.820884    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/747c597f-77cf-4287-a7a1-879565aec8ae-tls-certs\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.820979 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.820916    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-model-cache\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.891478 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.891436    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:36.904214 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.904190    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:36.921633 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.921606    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-home\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.921735 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.921650    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgbxm\" (UniqueName: \"kubernetes.io/projected/747c597f-77cf-4287-a7a1-879565aec8ae-kube-api-access-lgbxm\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.921735 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.921671    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-dshm\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.921735 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.921697    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/747c597f-77cf-4287-a7a1-879565aec8ae-tls-certs\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.921882 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.921742    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-model-cache\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.921882 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.921786    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-tmp-dir\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.922107 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.922081    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-home\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.922195 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.922170    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-tmp-dir\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.922507 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.922482    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-model-cache\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.924005 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.923977    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-dshm\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.924303 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.924284    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/747c597f-77cf-4287-a7a1-879565aec8ae-tls-certs\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:36.982245 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:36.982167    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgbxm\" (UniqueName: \"kubernetes.io/projected/747c597f-77cf-4287-a7a1-879565aec8ae-kube-api-access-lgbxm\") pod \"simulator-w4-ed0600-kserve-bf55d8dc4-6d59s\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") " pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:37.004770 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:37.004745    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:37.023931 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:37.023902    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:45:37.379723 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:37.379564    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"]
Nov 02 01:45:37.382324 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:45:37.382299    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod747c597f_77cf_4287_a7a1_879565aec8ae.slice/crio-ee236b03be60100cc1154a1150749a33b1326f0ce7bb5562fdbc56162f8eac6a WatchSource:0}: Error finding container ee236b03be60100cc1154a1150749a33b1326f0ce7bb5562fdbc56162f8eac6a: Status 404 returned error can't find the container with id ee236b03be60100cc1154a1150749a33b1326f0ce7bb5562fdbc56162f8eac6a
Nov 02 01:45:38.016778 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:38.016739    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" event={"ID":"747c597f-77cf-4287-a7a1-879565aec8ae","Type":"ContainerStarted","Data":"b1770a23cc5846868b62a1fe28b358cbbc99f9819f31acdeac0b2a7875ec6a1b"}
Nov 02 01:45:38.017200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:38.016788    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" event={"ID":"747c597f-77cf-4287-a7a1-879565aec8ae","Type":"ContainerStarted","Data":"ee236b03be60100cc1154a1150749a33b1326f0ce7bb5562fdbc56162f8eac6a"}
Nov 02 01:45:38.134265 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:38.134217    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" podStartSLOduration=2.134203816 podStartE2EDuration="2.134203816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:45:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:45:38.132585474 +0000 UTC m=+827.344955285" watchObservedRunningTime="2026-11-02 01:45:38.134203816 +0000 UTC m=+827.346573624"
Nov 02 01:45:39.005212 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:39.005181    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:39.958112 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:45:39.958076    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:45:47.005131 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:47.005096    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:47.017509 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:47.017479    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:47.065131 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:47.065082    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:45:59.496599 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.496569    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"]
Nov 02 01:45:59.501134 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.501101    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.602501 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.602467    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-9470a3\"/\"premium-w0-9470a3-kserve-self-signed-certs\""
Nov 02 01:45:59.611037 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.611007    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"]
Nov 02 01:45:59.627935 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.627905    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-home\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.628067 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.627945    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-dshm\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.628067 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.628058    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-tmp-dir\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.628175 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.628100    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-tls-certs\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.628175 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.628148    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-model-cache\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.628238 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.628201    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm5j8\" (UniqueName: \"kubernetes.io/projected/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-kube-api-access-cm5j8\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.729662 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.729629    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-tmp-dir\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.729851 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.729691    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-tls-certs\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.729851 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.729713    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-model-cache\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.729851 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.729736    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm5j8\" (UniqueName: \"kubernetes.io/projected/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-kube-api-access-cm5j8\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.729851 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.729762    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-home\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.729851 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.729788    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-dshm\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.730175 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.730149    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-tmp-dir\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.730263 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.730153    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-model-cache\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.730337 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.730313    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-home\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.732685 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.732659    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-dshm\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.732893 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.732875    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-tls-certs\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.790962 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.790885    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm5j8\" (UniqueName: \"kubernetes.io/projected/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-kube-api-access-cm5j8\") pod \"premium-w0-9470a3-kserve-64b6755468-rpxw5\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") " pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:45:59.812742 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:45:59.812712    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:46:00.196940 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:00.196909    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"]
Nov 02 01:46:00.199990 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:46:00.199962    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf03b6e9e_52c5_47cd_9de5_2073fc55f0d3.slice/crio-2f24820fea4503161952374c2901d5ad0c98259a1aaf31a4a4b33e77cb133bee WatchSource:0}: Error finding container 2f24820fea4503161952374c2901d5ad0c98259a1aaf31a4a4b33e77cb133bee: Status 404 returned error can't find the container with id 2f24820fea4503161952374c2901d5ad0c98259a1aaf31a4a4b33e77cb133bee
Nov 02 01:46:01.117482 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:01.117443    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" event={"ID":"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3","Type":"ContainerStarted","Data":"17b1db1f67528e21c6dbe6900d72aff00af48cf3f4fe83be0dfb5a0dbdd7d1ef"}
Nov 02 01:46:01.117482 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:01.117489    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" event={"ID":"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3","Type":"ContainerStarted","Data":"2f24820fea4503161952374c2901d5ad0c98259a1aaf31a4a4b33e77cb133bee"}
Nov 02 01:46:01.235825 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:01.235771    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" podStartSLOduration=2.235757376 podStartE2EDuration="2.235757376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:45:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:46:01.23429702 +0000 UTC m=+850.446666829" watchObservedRunningTime="2026-11-02 01:46:01.235757376 +0000 UTC m=+850.448127184"
Nov 02 01:46:01.813881 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:01.813838    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:46:09.486477 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.486396    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"]
Nov 02 01:46:09.489752 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.489736    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.518344 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.518308    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwph4\" (UniqueName: \"kubernetes.io/projected/73a93fb2-af95-4246-8032-e8ce2025bbdd-kube-api-access-kwph4\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.518510 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.518356    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73a93fb2-af95-4246-8032-e8ce2025bbdd-tls-certs\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.518579 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.518505    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-tmp-dir\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.518622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.518588    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-dshm\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.518622 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.518612    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-home\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.518702 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.518640    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-model-cache\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.545332 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.545293    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-c62671\"/\"default-dockercfg-z5z9w\""
Nov 02 01:46:09.594656 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.594627    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-c62671\"/\"openshift-service-ca.crt\""
Nov 02 01:46:09.594809 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.594666    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-c62671\"/\"kube-root-ca.crt\""
Nov 02 01:46:09.596337 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.596316    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"]
Nov 02 01:46:09.619360 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.619334    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwph4\" (UniqueName: \"kubernetes.io/projected/73a93fb2-af95-4246-8032-e8ce2025bbdd-kube-api-access-kwph4\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.619503 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.619367    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73a93fb2-af95-4246-8032-e8ce2025bbdd-tls-certs\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.619503 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.619455    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-tmp-dir\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.619618 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.619516    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-dshm\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.619618 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.619537    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-home\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.619618 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.619562    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-model-cache\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.620035 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.620011    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-model-cache\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.620159 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.620053    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-home\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.620211 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.620161    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-tmp-dir\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.621934 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.621913    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-dshm\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.645291 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.645270    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-c62671\"/\"simulator-w2-c62671-kserve-self-signed-certs\""
Nov 02 01:46:09.654014 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.653987    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73a93fb2-af95-4246-8032-e8ce2025bbdd-tls-certs\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.680231 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.680205    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwph4\" (UniqueName: \"kubernetes.io/projected/73a93fb2-af95-4246-8032-e8ce2025bbdd-kube-api-access-kwph4\") pod \"simulator-w2-c62671-kserve-78d86c9b54-rjh62\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") " pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.801879 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.801778    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:09.812970 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.812944    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:46:09.830095 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:09.830068    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:46:09.959404 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:46:09.959370    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:46:10.167896 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:10.167823    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:46:10.185681 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:10.185652    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"]
Nov 02 01:46:10.188570 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:46:10.188546    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73a93fb2_af95_4246_8032_e8ce2025bbdd.slice/crio-2d4671f6292d5a6188e739380e13ffeea2841cc5d1f409987fd2bea481415614 WatchSource:0}: Error finding container 2d4671f6292d5a6188e739380e13ffeea2841cc5d1f409987fd2bea481415614: Status 404 returned error can't find the container with id 2d4671f6292d5a6188e739380e13ffeea2841cc5d1f409987fd2bea481415614
Nov 02 01:46:11.163157 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:11.163102    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" event={"ID":"73a93fb2-af95-4246-8032-e8ce2025bbdd","Type":"ContainerStarted","Data":"e66c14ccb9856632233bab056368e61a0b3a4bed1e269e293a44e5e2b2b2e900"}
Nov 02 01:46:11.163692 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:11.163166    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" event={"ID":"73a93fb2-af95-4246-8032-e8ce2025bbdd","Type":"ContainerStarted","Data":"2d4671f6292d5a6188e739380e13ffeea2841cc5d1f409987fd2bea481415614"}
Nov 02 01:46:11.280081 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:11.280028    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" podStartSLOduration=2.28001146 podStartE2EDuration="2.28001146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:46:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:46:11.278717949 +0000 UTC m=+860.491087758" watchObservedRunningTime="2026-11-02 01:46:11.28001146 +0000 UTC m=+860.492381307"
Nov 02 01:46:11.802798 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:11.802770    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:19.801941 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:19.801903    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:19.814881 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:19.814855    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:20.214706 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:20.214678    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:46:28.297003 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.296962    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"]
Nov 02 01:46:28.300724 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.300708    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.398701 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.398667    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt5mv\" (UniqueName: \"kubernetes.io/projected/b1893195-677a-4c3d-8058-7623c4505f3d-kube-api-access-vt5mv\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.398701 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.398703    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1893195-677a-4c3d-8058-7623c4505f3d-tls-certs\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.398907 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.398795    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-model-cache\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.398907 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.398895    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-home\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.398971 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.398915    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-dshm\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.399011 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.398999    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-tmp-dir\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.402323 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.402302    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-55bc84\"/\"premium-w5-55bc84-kserve-self-signed-certs\""
Nov 02 01:46:28.403961 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.403941    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"]
Nov 02 01:46:28.499715 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.499672    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt5mv\" (UniqueName: \"kubernetes.io/projected/b1893195-677a-4c3d-8058-7623c4505f3d-kube-api-access-vt5mv\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.499715 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.499719    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1893195-677a-4c3d-8058-7623c4505f3d-tls-certs\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.499966 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.499772    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-model-cache\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.499966 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.499861    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-home\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.499966 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.499885    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-dshm\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.499966 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.499936    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-tmp-dir\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.500334 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.500310    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-home\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.500524 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.500364    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-model-cache\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.500524 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.500380    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-tmp-dir\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.502230 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.502209    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-dshm\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.502309 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.502294    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1893195-677a-4c3d-8058-7623c4505f3d-tls-certs\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.560528 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.560447    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt5mv\" (UniqueName: \"kubernetes.io/projected/b1893195-677a-4c3d-8058-7623c4505f3d-kube-api-access-vt5mv\") pod \"premium-w5-55bc84-kserve-69db7c486-xbcvz\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") " pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.612944 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.612904    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:28.993181 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:28.993149    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"]
Nov 02 01:46:28.996038 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:46:28.996010    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1893195_677a_4c3d_8058_7623c4505f3d.slice/crio-7669e7540f561b7fb5799ca0c9116597f1d43e19e54c1f3636d5c2f938000d2d WatchSource:0}: Error finding container 7669e7540f561b7fb5799ca0c9116597f1d43e19e54c1f3636d5c2f938000d2d: Status 404 returned error can't find the container with id 7669e7540f561b7fb5799ca0c9116597f1d43e19e54c1f3636d5c2f938000d2d
Nov 02 01:46:29.253072 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:29.252985    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" event={"ID":"b1893195-677a-4c3d-8058-7623c4505f3d","Type":"ContainerStarted","Data":"4972f69a550bea27d5c832d23ed6cd06fea22f142ab9170e954276b1bd756b14"}
Nov 02 01:46:29.253072 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:29.253036    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" event={"ID":"b1893195-677a-4c3d-8058-7623c4505f3d","Type":"ContainerStarted","Data":"7669e7540f561b7fb5799ca0c9116597f1d43e19e54c1f3636d5c2f938000d2d"}
Nov 02 01:46:29.369105 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:29.369049    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" podStartSLOduration=1.369035703 podStartE2EDuration="1.369035703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:46:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:46:29.3685541 +0000 UTC m=+878.580923909" watchObservedRunningTime="2026-11-02 01:46:29.369035703 +0000 UTC m=+878.581405510"
Nov 02 01:46:29.613877 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:29.613800    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:38.613029 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:38.612994    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:38.626107 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:38.626084    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:39.308516 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:39.308485    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:46:39.958560 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:46:39.958526    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:46:51.031140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:51.031096    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 01:46:51.031832 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:46:51.031810    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 01:47:09.958369 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:47:09.958333    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:47:39.957912 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:47:39.957873    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:47:48.989479 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:48.989443    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"]
Nov 02 01:47:48.992897 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:48.992880    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.066254 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.066219    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-model-cache\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.066426 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.066286    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-tmp-dir\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.066426 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.066385    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc4865ed-8a4f-4432-afc1-61d816b83090-tls-certs\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.066554 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.066527    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxpvr\" (UniqueName: \"kubernetes.io/projected/dc4865ed-8a4f-4432-afc1-61d816b83090-kube-api-access-sxpvr\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.066645 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.066628    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-dshm\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.066698 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.066663    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-home\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.094310 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.094285    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-c62671\"/\"premium-w2-c62671-kserve-self-signed-certs\""
Nov 02 01:47:49.096264 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.096244    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"]
Nov 02 01:47:49.168491 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.168463    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-tmp-dir\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.168683 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.168500    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc4865ed-8a4f-4432-afc1-61d816b83090-tls-certs\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.168683 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.168552    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxpvr\" (UniqueName: \"kubernetes.io/projected/dc4865ed-8a4f-4432-afc1-61d816b83090-kube-api-access-sxpvr\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.168683 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.168590    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-dshm\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.168683 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.168608    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-home\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.168932 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.168744    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-model-cache\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.168994 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.168926    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-tmp-dir\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.168994 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.168975    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-model-cache\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.169090 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.169043    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-home\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.170817 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.170784    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-dshm\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.171132 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.171101    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc4865ed-8a4f-4432-afc1-61d816b83090-tls-certs\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.229567 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.229537    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxpvr\" (UniqueName: \"kubernetes.io/projected/dc4865ed-8a4f-4432-afc1-61d816b83090-kube-api-access-sxpvr\") pod \"premium-w2-c62671-kserve-74bcdbcf87-57wvc\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") " pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.303948 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.303864    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:49.680152 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:49.680058    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"]
Nov 02 01:47:49.682729 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:47:49.682699    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc4865ed_8a4f_4432_afc1_61d816b83090.slice/crio-c2ca3b6a2a5168f78499d83fbd4c0cb8c7f101a202679373394916cf61d8cf43 WatchSource:0}: Error finding container c2ca3b6a2a5168f78499d83fbd4c0cb8c7f101a202679373394916cf61d8cf43: Status 404 returned error can't find the container with id c2ca3b6a2a5168f78499d83fbd4c0cb8c7f101a202679373394916cf61d8cf43
Nov 02 01:47:50.626661 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:50.626623    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc" event={"ID":"dc4865ed-8a4f-4432-afc1-61d816b83090","Type":"ContainerStarted","Data":"4da28877e92beda49c4caf64ecca0f592ea37b2bd6960ea3e148d19bc8aa08bb"}
Nov 02 01:47:50.627057 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:50.626673    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc" event={"ID":"dc4865ed-8a4f-4432-afc1-61d816b83090","Type":"ContainerStarted","Data":"c2ca3b6a2a5168f78499d83fbd4c0cb8c7f101a202679373394916cf61d8cf43"}
Nov 02 01:47:50.743897 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:50.743847    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc" podStartSLOduration=2.743833673 podStartE2EDuration="2.743833673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:47:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:47:50.742467026 +0000 UTC m=+959.954836834" watchObservedRunningTime="2026-11-02 01:47:50.743833673 +0000 UTC m=+959.956203478"
Nov 02 01:47:51.304252 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:51.304216    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:59.304583 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:59.304547    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:59.318873 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:59.318849    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:47:59.679348 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:47:59.679277    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:48:09.958448 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:48:09.958409    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:48:26.825009 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:26.824979    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"]
Nov 02 01:48:26.825573 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:26.825190    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49" podUID="1e6f4951-bf67-4011-ad79-e02a68472025" containerName="istio-proxy" containerID="cri-o://b1184a9e29db854e423d4ce2042ae468789b4875583eba32d89bdf3448002456" gracePeriod=30
Nov 02 01:48:32.072138 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.072101    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:48:32.182888 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.182808    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e6f4951-bf67-4011-ad79-e02a68472025-istio-token\" (UniqueName: \"kubernetes.io/projected/1e6f4951-bf67-4011-ad79-e02a68472025-istio-token\") pod \"1e6f4951-bf67-4011-ad79-e02a68472025\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") "
Nov 02 01:48:32.182888 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.182842    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1e6f4951-bf67-4011-ad79-e02a68472025-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e6f4951-bf67-4011-ad79-e02a68472025-istiod-ca-cert\") pod \"1e6f4951-bf67-4011-ad79-e02a68472025\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") "
Nov 02 01:48:32.183097 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.182950    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-workload-socket\") pod \"1e6f4951-bf67-4011-ad79-e02a68472025\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") "
Nov 02 01:48:32.183097 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.182976    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-workload-certs\") pod \"1e6f4951-bf67-4011-ad79-e02a68472025\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") "
Nov 02 01:48:32.183097 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.182996    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-istio-envoy\") pod \"1e6f4951-bf67-4011-ad79-e02a68472025\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") "
Nov 02 01:48:32.183097 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.183019    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e6f4951-bf67-4011-ad79-e02a68472025-kube-api-access-gp6dv\" (UniqueName: \"kubernetes.io/projected/1e6f4951-bf67-4011-ad79-e02a68472025-kube-api-access-gp6dv\") pod \"1e6f4951-bf67-4011-ad79-e02a68472025\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") "
Nov 02 01:48:32.183097 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.183097    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-credential-socket\") pod \"1e6f4951-bf67-4011-ad79-e02a68472025\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") "
Nov 02 01:48:32.183452 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.183153    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1e6f4951-bf67-4011-ad79-e02a68472025-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e6f4951-bf67-4011-ad79-e02a68472025-istio-podinfo\") pod \"1e6f4951-bf67-4011-ad79-e02a68472025\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") "
Nov 02 01:48:32.183452 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.183182    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-istio-data\") pod \"1e6f4951-bf67-4011-ad79-e02a68472025\" (UID: \"1e6f4951-bf67-4011-ad79-e02a68472025\") "
Nov 02 01:48:32.183452 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.183330    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-workload-socket" pod "1e6f4951-bf67-4011-ad79-e02a68472025" (UID: "1e6f4951-bf67-4011-ad79-e02a68472025"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:48:32.183452 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.183380    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-workload-certs" pod "1e6f4951-bf67-4011-ad79-e02a68472025" (UID: "1e6f4951-bf67-4011-ad79-e02a68472025"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:48:32.183452 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.183406    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e6f4951-bf67-4011-ad79-e02a68472025-istiod-ca-cert" pod "1e6f4951-bf67-4011-ad79-e02a68472025" (UID: "1e6f4951-bf67-4011-ad79-e02a68472025"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:48:32.183715 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.183640    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:48:32.183715 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.183660    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:48:32.183715 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.183675    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e6f4951-bf67-4011-ad79-e02a68472025-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:48:32.183715 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.183651    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-credential-socket" pod "1e6f4951-bf67-4011-ad79-e02a68472025" (UID: "1e6f4951-bf67-4011-ad79-e02a68472025"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:48:32.183715 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.183667    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-istio-data" pod "1e6f4951-bf67-4011-ad79-e02a68472025" (UID: "1e6f4951-bf67-4011-ad79-e02a68472025"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:48:32.185682 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.185659    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e6f4951-bf67-4011-ad79-e02a68472025-istio-token" pod "1e6f4951-bf67-4011-ad79-e02a68472025" (UID: "1e6f4951-bf67-4011-ad79-e02a68472025"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 01:48:32.185786 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.185693    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1e6f4951-bf67-4011-ad79-e02a68472025-istio-podinfo" pod "1e6f4951-bf67-4011-ad79-e02a68472025" (UID: "1e6f4951-bf67-4011-ad79-e02a68472025"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 01:48:32.185786 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.185717    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e6f4951-bf67-4011-ad79-e02a68472025-kube-api-access-gp6dv" pod "1e6f4951-bf67-4011-ad79-e02a68472025" (UID: "1e6f4951-bf67-4011-ad79-e02a68472025"). InnerVolumeSpecName "kube-api-access-gp6dv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 01:48:32.185786 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.185729    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-istio-envoy" pod "1e6f4951-bf67-4011-ad79-e02a68472025" (UID: "1e6f4951-bf67-4011-ad79-e02a68472025"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:48:32.285132 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.285092    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e6f4951-bf67-4011-ad79-e02a68472025-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:48:32.285132 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.285131    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:48:32.285338 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.285145    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp6dv\" (UniqueName: \"kubernetes.io/projected/1e6f4951-bf67-4011-ad79-e02a68472025-kube-api-access-gp6dv\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:48:32.285338 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.285160    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:48:32.285338 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.285172    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e6f4951-bf67-4011-ad79-e02a68472025-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:48:32.285338 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.285185    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e6f4951-bf67-4011-ad79-e02a68472025-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:48:32.813131 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.813090    4515 generic.go:356] "Generic (PLEG): container finished" podID="1e6f4951-bf67-4011-ad79-e02a68472025" containerID="b1184a9e29db854e423d4ce2042ae468789b4875583eba32d89bdf3448002456" exitCode=0
Nov 02 01:48:32.813304 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.813150    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49" event={"ID":"1e6f4951-bf67-4011-ad79-e02a68472025","Type":"ContainerDied","Data":"b1184a9e29db854e423d4ce2042ae468789b4875583eba32d89bdf3448002456"}
Nov 02 01:48:32.813304 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.813162    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"
Nov 02 01:48:32.813304 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.813240    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49" event={"ID":"1e6f4951-bf67-4011-ad79-e02a68472025","Type":"ContainerDied","Data":"579d86bb682df86aa9e7b9b93d30f9d36636e8e8f39b5b1c87e550ee452773d1"}
Nov 02 01:48:32.813304 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.813260    4515 scope.go:122] "RemoveContainer" containerID="b1184a9e29db854e423d4ce2042ae468789b4875583eba32d89bdf3448002456"
Nov 02 01:48:32.824587 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.824567    4515 scope.go:122] "RemoveContainer" containerID="b1184a9e29db854e423d4ce2042ae468789b4875583eba32d89bdf3448002456"
Nov 02 01:48:32.824871 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:48:32.824852    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1184a9e29db854e423d4ce2042ae468789b4875583eba32d89bdf3448002456\": container with ID starting with b1184a9e29db854e423d4ce2042ae468789b4875583eba32d89bdf3448002456 not found: ID does not exist" containerID="b1184a9e29db854e423d4ce2042ae468789b4875583eba32d89bdf3448002456"
Nov 02 01:48:32.824929 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:32.824883    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1184a9e29db854e423d4ce2042ae468789b4875583eba32d89bdf3448002456"} err="failed to get container status \"b1184a9e29db854e423d4ce2042ae468789b4875583eba32d89bdf3448002456\": rpc error: code = NotFound desc = could not find container \"b1184a9e29db854e423d4ce2042ae468789b4875583eba32d89bdf3448002456\": container with ID starting with b1184a9e29db854e423d4ce2042ae468789b4875583eba32d89bdf3448002456 not found: ID does not exist"
Nov 02 01:48:33.027523 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:33.027481    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"]
Nov 02 01:48:33.077483 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:33.077414    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-n4f49"]
Nov 02 01:48:34.952102 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:34.952071    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e6f4951-bf67-4011-ad79-e02a68472025" path="/var/lib/kubelet/pods/1e6f4951-bf67-4011-ad79-e02a68472025/volumes"
Nov 02 01:48:39.959093 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:48:39.959055    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:48:52.072500 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.072469    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"]
Nov 02 01:48:52.072884 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.072814    4515 cpu_manager.go:414] "containerMap: removing container" podUID="1e6f4951-bf67-4011-ad79-e02a68472025" containerName="istio-proxy"
Nov 02 01:48:52.072884 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.072824    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e6f4951-bf67-4011-ad79-e02a68472025" containerName="istio-proxy"
Nov 02 01:48:52.072958 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.072892    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e6f4951-bf67-4011-ad79-e02a68472025" containerName="istio-proxy"
Nov 02 01:48:52.076933 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.076914    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.129077 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.129040    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-061fab-openshift-default-dockercfg-k8pns\""
Nov 02 01:48:52.171421 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.171383    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-credential-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.171586 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.171432    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-istio-envoy\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.171586 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.171528    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6btzh\" (UniqueName: \"kubernetes.io/projected/47069c5d-070f-425b-ba94-a69d0585fd29-kube-api-access-6btzh\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.171586 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.171565    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-workload-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.171703 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.171637    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47069c5d-070f-425b-ba94-a69d0585fd29-istiod-ca-cert\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.171703 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.171672    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-istio-data\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.171703 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.171692    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47069c5d-070f-425b-ba94-a69d0585fd29-istio-podinfo\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.171846 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.171718    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47069c5d-070f-425b-ba94-a69d0585fd29-istio-token\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.171904 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.171873    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-workload-certs\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.180523 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.180498    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"]
Nov 02 01:48:52.273214 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.273180    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-workload-certs\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.273427 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.273237    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-credential-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.273427 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.273269    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-istio-envoy\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.273427 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.273314    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6btzh\" (UniqueName: \"kubernetes.io/projected/47069c5d-070f-425b-ba94-a69d0585fd29-kube-api-access-6btzh\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.273427 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.273345    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-workload-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.273427 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.273391    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47069c5d-070f-425b-ba94-a69d0585fd29-istiod-ca-cert\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.273427 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.273414    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-istio-data\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.273718 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.273436    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47069c5d-070f-425b-ba94-a69d0585fd29-istio-podinfo\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.273718 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.273467    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47069c5d-070f-425b-ba94-a69d0585fd29-istio-token\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.273819 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.273728    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-credential-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.273876 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.273829    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-istio-data\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.273930 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.273884    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-workload-certs\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.273930 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.273916    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-workload-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.274171 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.274136    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47069c5d-070f-425b-ba94-a69d0585fd29-istiod-ca-cert\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.275966 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.275939    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-istio-envoy\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.276171 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.276154    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47069c5d-070f-425b-ba94-a69d0585fd29-istio-podinfo\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.333978 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.333911    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47069c5d-070f-425b-ba94-a69d0585fd29-istio-token\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.334103 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.334033    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6btzh\" (UniqueName: \"kubernetes.io/projected/47069c5d-070f-425b-ba94-a69d0585fd29-kube-api-access-6btzh\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.389426 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.389396    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:52.814535 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.814504    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"]
Nov 02 01:48:52.817501 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:48:52.817472    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47069c5d_070f_425b_ba94_a69d0585fd29.slice/crio-7b8701b5f13b32baa32ddd5fc983de123f61c44a6f7e64992c291547b33fbfa1 WatchSource:0}: Error finding container 7b8701b5f13b32baa32ddd5fc983de123f61c44a6f7e64992c291547b33fbfa1: Status 404 returned error can't find the container with id 7b8701b5f13b32baa32ddd5fc983de123f61c44a6f7e64992c291547b33fbfa1
Nov 02 01:48:52.819749 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.819720    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:48:52.819814 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.819782    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:48:52.819814 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.819811    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:48:52.912141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.912092    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np" event={"ID":"47069c5d-070f-425b-ba94-a69d0585fd29","Type":"ContainerStarted","Data":"39e46d97ddf088ca52e1e2bfa0ffdcf7a9daa2208b005b62bd1b3618aa3f5db1"}
Nov 02 01:48:52.912268 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:52.912168    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np" event={"ID":"47069c5d-070f-425b-ba94-a69d0585fd29","Type":"ContainerStarted","Data":"7b8701b5f13b32baa32ddd5fc983de123f61c44a6f7e64992c291547b33fbfa1"}
Nov 02 01:48:53.032095 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:53.032045    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np" podStartSLOduration=1.032031478 podStartE2EDuration="1.032031478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:48:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:48:53.030923239 +0000 UTC m=+1022.243293071" watchObservedRunningTime="2026-11-02 01:48:53.032031478 +0000 UTC m=+1022.244401347"
Nov 02 01:48:53.389762 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:53.389724    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:53.391300 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:53.391269    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 02 01:48:53.391423 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:53.391332    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np" podUID="47069c5d-070f-425b-ba94-a69d0585fd29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 02 01:48:54.389942 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:54.389906    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 02 01:48:54.390396 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:54.389958    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np" podUID="47069c5d-070f-425b-ba94-a69d0585fd29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 02 01:48:55.389923 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:55.389882    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 02 01:48:55.390140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:55.389936    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np" podUID="47069c5d-070f-425b-ba94-a69d0585fd29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 02 01:48:56.390319 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:56.390285    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 02 01:48:56.390795 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:56.390347    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np" podUID="47069c5d-070f-425b-ba94-a69d0585fd29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 02 01:48:57.390015 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:57.389975    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 02 01:48:57.390227 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:57.390039    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np" podUID="47069c5d-070f-425b-ba94-a69d0585fd29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 02 01:48:58.390404 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:58.390357    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 02 01:48:58.390958 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:58.390426    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np" podUID="47069c5d-070f-425b-ba94-a69d0585fd29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 02 01:48:59.511168 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:59.511136    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:59.511965 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:59.511919    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:48:59.512841 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:48:59.512821    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 01:49:09.958292 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:49:09.958207    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:49:27.313804 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.313771    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"]
Nov 02 01:49:27.317528 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.317507    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.420514 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.420482    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"]
Nov 02 01:49:27.427289 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.427249    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.427447 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.427371    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.427447 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.427427    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.427538 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.427457    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.427538 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.427483    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twxvt\" (UniqueName: \"kubernetes.io/projected/6cbffd1e-1b76-472b-b656-e2df763f243a-kube-api-access-twxvt\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.427637 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.427558    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.427637 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.427622    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.427737 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.427700    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.427813 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.427788    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cbffd1e-1b76-472b-b656-e2df763f243a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.529351 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.529317    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.529549 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.529366    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.529549 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.529398    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.529549 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.529415    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twxvt\" (UniqueName: \"kubernetes.io/projected/6cbffd1e-1b76-472b-b656-e2df763f243a-kube-api-access-twxvt\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.529549 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.529447    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.529549 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.529472    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.529549 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.529534    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.529867 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.529559    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cbffd1e-1b76-472b-b656-e2df763f243a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.529867 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.529616    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.529867 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.529763    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.529867 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.529835    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.530035 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.529936    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.530035 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.529986    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.530386 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.530365    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cbffd1e-1b76-472b-b656-e2df763f243a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.531813 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.531792    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.531932 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.531914    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.592278 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.592200    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.592434 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.592405    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twxvt\" (UniqueName: \"kubernetes.io/projected/6cbffd1e-1b76-472b-b656-e2df763f243a-kube-api-access-twxvt\") pod \"maas-default-gateway-openshift-default-778b7b5c47-7w44z\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:27.630446 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:27.630411    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:28.061243 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:28.061222    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"]
Nov 02 01:49:28.063802 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:49:28.063776    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cbffd1e_1b76_472b_b656_e2df763f243a.slice/crio-3f93700b18cd9e3b6d51cf137cc482179f630853a573e1f93ed2250a682cb611 WatchSource:0}: Error finding container 3f93700b18cd9e3b6d51cf137cc482179f630853a573e1f93ed2250a682cb611: Status 404 returned error can't find the container with id 3f93700b18cd9e3b6d51cf137cc482179f630853a573e1f93ed2250a682cb611
Nov 02 01:49:28.066741 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:28.066707    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:49:28.066827 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:28.066778    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:49:28.066827 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:28.066808    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:49:28.116609 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:28.116582    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z" event={"ID":"6cbffd1e-1b76-472b-b656-e2df763f243a","Type":"ContainerStarted","Data":"3f93700b18cd9e3b6d51cf137cc482179f630853a573e1f93ed2250a682cb611"}
Nov 02 01:49:29.123730 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:29.123691    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z" event={"ID":"6cbffd1e-1b76-472b-b656-e2df763f243a","Type":"ContainerStarted","Data":"379293b0c90404c195e48a011478162f9f37b3496feea7e20fa5dba50605fe40"}
Nov 02 01:49:29.314296 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:29.314241    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z" podStartSLOduration=2.314226432 podStartE2EDuration="2.314226432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:49:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:49:29.312808963 +0000 UTC m=+1058.525178794" watchObservedRunningTime="2026-11-02 01:49:29.314226432 +0000 UTC m=+1058.526596239"
Nov 02 01:49:29.647146 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:29.646349    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:29.647146 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:29.646475    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-7w44z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Nov 02 01:49:29.647146 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:29.646517    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z" podUID="6cbffd1e-1b76-472b-b656-e2df763f243a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Nov 02 01:49:30.646508 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:30.646467    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-7w44z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Nov 02 01:49:30.647096 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:30.647063    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z" podUID="6cbffd1e-1b76-472b-b656-e2df763f243a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Nov 02 01:49:31.640141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:31.636917    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-7w44z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Nov 02 01:49:31.640141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:31.636999    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z" podUID="6cbffd1e-1b76-472b-b656-e2df763f243a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Nov 02 01:49:32.647140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:32.646562    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-7w44z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Nov 02 01:49:32.647140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:32.646640    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z" podUID="6cbffd1e-1b76-472b-b656-e2df763f243a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Nov 02 01:49:33.647922 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:33.647884    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-7w44z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Nov 02 01:49:33.648537 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:33.648486    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z" podUID="6cbffd1e-1b76-472b-b656-e2df763f243a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Nov 02 01:49:34.631300 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.631260    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-7w44z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Nov 02 01:49:34.631510 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.631322    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z" podUID="6cbffd1e-1b76-472b-b656-e2df763f243a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Nov 02 01:49:34.709309 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.709266    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"]
Nov 02 01:49:34.713638 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.713613    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.765969 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.765941    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-9470a3-openshift-default-dockercfg-pfjf7\""
Nov 02 01:49:34.809787 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.809741    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-workload-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.809787 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.809784    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-token\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.810050 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.809818    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-workload-certs\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.810050 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.809977    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-envoy\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.810050 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.810000    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-data\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.810050 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.810042    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-podinfo\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.810317 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.810105    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce83f3e-50cb-44f6-8c6c-4de081152950-istiod-ca-cert\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.810317 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.810193    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvssk\" (UniqueName: \"kubernetes.io/projected/7ce83f3e-50cb-44f6-8c6c-4de081152950-kube-api-access-rvssk\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.810317 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.810253    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-credential-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.818811 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.818781    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"]
Nov 02 01:49:34.911668 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.911575    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-envoy\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.911668 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.911626    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-data\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.911904 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.911680    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-podinfo\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.911904 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.911736    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce83f3e-50cb-44f6-8c6c-4de081152950-istiod-ca-cert\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.911904 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.911766    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvssk\" (UniqueName: \"kubernetes.io/projected/7ce83f3e-50cb-44f6-8c6c-4de081152950-kube-api-access-rvssk\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.911904 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.911818    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-credential-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.911904 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.911864    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-workload-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.911904 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.911899    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-token\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.912253 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.911954    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-workload-certs\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.912253 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.912161    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-data\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.912534 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.912497    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-workload-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.912670 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.912628    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-workload-certs\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.912670 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.912658    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce83f3e-50cb-44f6-8c6c-4de081152950-istiod-ca-cert\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.912803 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.912684    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-credential-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.914245 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.914219    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-envoy\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.914366 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.914303    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-podinfo\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.972260 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.972232    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-token\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:34.972590 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:34.972550    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvssk\" (UniqueName: \"kubernetes.io/projected/7ce83f3e-50cb-44f6-8c6c-4de081152950-kube-api-access-rvssk\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:35.027386 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:35.027355    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:35.462359 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:35.462334    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"]
Nov 02 01:49:35.464687 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:49:35.464649    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce83f3e_50cb_44f6_8c6c_4de081152950.slice/crio-60e7051f63284b5ba16b15f2a84f30a4203fe207f3673a451cb538543a74a32d WatchSource:0}: Error finding container 60e7051f63284b5ba16b15f2a84f30a4203fe207f3673a451cb538543a74a32d: Status 404 returned error can't find the container with id 60e7051f63284b5ba16b15f2a84f30a4203fe207f3673a451cb538543a74a32d
Nov 02 01:49:35.467246 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:35.467200    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:49:35.467324 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:35.467307    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:49:35.467365 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:35.467341    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:49:36.182494 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:36.182455    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj" event={"ID":"7ce83f3e-50cb-44f6-8c6c-4de081152950","Type":"ContainerStarted","Data":"4b964a75fed3cdd9c401ca110557b51f7fed522d9a4c33a99b8b1904a73c5a8f"}
Nov 02 01:49:36.182924 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:36.182522    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj" event={"ID":"7ce83f3e-50cb-44f6-8c6c-4de081152950","Type":"ContainerStarted","Data":"60e7051f63284b5ba16b15f2a84f30a4203fe207f3673a451cb538543a74a32d"}
Nov 02 01:49:36.405453 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:36.405396    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj" podStartSLOduration=2.40537945 podStartE2EDuration="2.40537945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:49:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:49:36.353610489 +0000 UTC m=+1065.565980300" watchObservedRunningTime="2026-11-02 01:49:36.40537945 +0000 UTC m=+1065.617749257"
Nov 02 01:49:36.419565 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:36.419534    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:36.420482 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:36.420456    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:36.421669 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:36.421649    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:49:37.048832 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:37.048792    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:37.050469 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:37.050433    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 02 01:49:37.050610 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:37.050491    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj" podUID="7ce83f3e-50cb-44f6-8c6c-4de081152950" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 02 01:49:37.773759 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:37.773723    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"]
Nov 02 01:49:37.777240 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:37.777222    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:37.838163 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:37.838131    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-55bc84-openshift-default-dockercfg-7g4tr\""
Nov 02 01:49:37.899954 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:37.899916    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"]
Nov 02 01:49:37.961744 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:37.961704    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-workload-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:37.961744 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:37.961742    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/444174f7-6dd7-411d-b625-e3edd0f214ce-istiod-ca-cert\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:37.961964 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:37.961766    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-workload-certs\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:37.961964 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:37.961913    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-data\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:37.962030 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:37.961974    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-credential-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:37.962030 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:37.962005    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-token\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:37.962030 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:37.962025    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjrgc\" (UniqueName: \"kubernetes.io/projected/444174f7-6dd7-411d-b625-e3edd0f214ce-kube-api-access-qjrgc\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:37.962161 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:37.962042    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-podinfo\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:37.962206 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:37.962171    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-envoy\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.041579 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.041485    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 02 01:49:38.041832 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.041804    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj" podUID="7ce83f3e-50cb-44f6-8c6c-4de081152950" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 02 01:49:38.062920 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.062886    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-credential-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.062920 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.062927    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-token\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.063189 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.062946    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjrgc\" (UniqueName: \"kubernetes.io/projected/444174f7-6dd7-411d-b625-e3edd0f214ce-kube-api-access-qjrgc\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.063189 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.062994    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-podinfo\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.063189 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.063061    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-envoy\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.063189 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.063145    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-workload-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.063189 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.063170    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/444174f7-6dd7-411d-b625-e3edd0f214ce-istiod-ca-cert\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.063472 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.063204    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-workload-certs\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.063472 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.063295    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-data\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.063573 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.063490    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-credential-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.064008 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.063707    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-data\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.064008 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.063873    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-workload-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.064008 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.063959    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-workload-certs\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.064240 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.064148    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/444174f7-6dd7-411d-b625-e3edd0f214ce-istiod-ca-cert\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.065641 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.065605    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-envoy\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.065745 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.065730    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-podinfo\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.138145 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.124833    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjrgc\" (UniqueName: \"kubernetes.io/projected/444174f7-6dd7-411d-b625-e3edd0f214ce-kube-api-access-qjrgc\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.140382 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.140324    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-token\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.390642 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.390553    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:38.857564 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.853479    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"]
Nov 02 01:49:38.861178 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:49:38.859678    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod444174f7_6dd7_411d_b625_e3edd0f214ce.slice/crio-28c394a1bafeb5aacf173f2ed7d7d6a3ccc9b47cadaf17d3d110896cf78baeda WatchSource:0}: Error finding container 28c394a1bafeb5aacf173f2ed7d7d6a3ccc9b47cadaf17d3d110896cf78baeda: Status 404 returned error can't find the container with id 28c394a1bafeb5aacf173f2ed7d7d6a3ccc9b47cadaf17d3d110896cf78baeda
Nov 02 01:49:38.862746 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.862708    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:49:38.862852 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.862835    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:49:38.862900 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:38.862883    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:49:39.030645 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:39.030611    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 02 01:49:39.030902 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:39.030873    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj" podUID="7ce83f3e-50cb-44f6-8c6c-4de081152950" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 02 01:49:39.198360 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:39.198325    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx" event={"ID":"444174f7-6dd7-411d-b625-e3edd0f214ce","Type":"ContainerStarted","Data":"67d599c809cf9b757f0ab84bf60b9d4172ba11a6071d9fddae72a551bfad3c18"}
Nov 02 01:49:39.198576 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:39.198381    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx" event={"ID":"444174f7-6dd7-411d-b625-e3edd0f214ce","Type":"ContainerStarted","Data":"28c394a1bafeb5aacf173f2ed7d7d6a3ccc9b47cadaf17d3d110896cf78baeda"}
Nov 02 01:49:39.317787 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:39.317730    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx" podStartSLOduration=2.317710552 podStartE2EDuration="2.317710552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:49:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:49:39.316414046 +0000 UTC m=+1068.528783864" watchObservedRunningTime="2026-11-02 01:49:39.317710552 +0000 UTC m=+1068.530080361"
Nov 02 01:49:39.391132 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:39.391091    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:39.392629 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:39.392606    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 02 01:49:39.392756 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:39.392649    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx" podUID="444174f7-6dd7-411d-b625-e3edd0f214ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 02 01:49:39.995525 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:49:39.995485    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:49:40.039141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:40.037684    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 02 01:49:40.039141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:40.037761    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj" podUID="7ce83f3e-50cb-44f6-8c6c-4de081152950" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 02 01:49:40.392200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:40.391728    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 02 01:49:40.392200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:40.391777    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx" podUID="444174f7-6dd7-411d-b625-e3edd0f214ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 02 01:49:41.032600 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:41.032568    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 02 01:49:41.033227 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:41.033169    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj" podUID="7ce83f3e-50cb-44f6-8c6c-4de081152950" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 02 01:49:41.391597 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:41.391512    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 02 01:49:41.391597 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:41.391573    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx" podUID="444174f7-6dd7-411d-b625-e3edd0f214ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 02 01:49:42.028476 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:42.028438    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 02 01:49:42.028749 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:42.028721    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj" podUID="7ce83f3e-50cb-44f6-8c6c-4de081152950" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 02 01:49:42.391999 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:42.391916    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 02 01:49:42.391999 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:42.391972    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx" podUID="444174f7-6dd7-411d-b625-e3edd0f214ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 02 01:49:43.390999 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:43.390957    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 02 01:49:43.391241 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:43.391009    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx" podUID="444174f7-6dd7-411d-b625-e3edd0f214ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 02 01:49:43.683066 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:43.682989    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:43.683444 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:43.683355    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:43.684111 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:43.684090    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 01:49:44.391865 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:44.391834    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 02 01:49:44.392051 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:44.391884    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx" podUID="444174f7-6dd7-411d-b625-e3edd0f214ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 02 01:49:45.569054 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:45.569019    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:45.569596 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:45.569573    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:49:45.570410 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:49:45.570390    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 01:50:08.613450 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.613416    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"]
Nov 02 01:50:08.621423 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.619481    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.672694 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.672672    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-dbe849-openshift-default-dockercfg-7b7d6\""
Nov 02 01:50:08.697066 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.697043    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fdgr\" (UniqueName: \"kubernetes.io/projected/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-kube-api-access-4fdgr\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.697225 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.697092    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-podinfo\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.697225 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.697136    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-token\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.697342 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.697237    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-data\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.697342 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.697274    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-credential-socket\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.697342 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.697308    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-envoy\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.697470 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.697384    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istiod-ca-cert\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.697470 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.697409    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-workload-certs\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.697470 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.697453    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-workload-socket\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.725164 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.725135    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"]
Nov 02 01:50:08.798875 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.798844    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-podinfo\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.799063 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.798893    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-token\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.799063 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.798921    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-data\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.799063 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.798944    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-credential-socket\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.799063 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.798966    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-envoy\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.799063 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.799026    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istiod-ca-cert\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.799369 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.799064    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-workload-certs\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.799369 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.799166    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-workload-socket\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.799369 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.799233    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fdgr\" (UniqueName: \"kubernetes.io/projected/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-kube-api-access-4fdgr\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.799515 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.799427    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-credential-socket\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.799598 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.799575    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-workload-socket\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.799661 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.799637    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-workload-certs\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.799712 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.799658    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-data\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.799803 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.799780    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istiod-ca-cert\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.801370 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.801347    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-envoy\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.801636 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.801619    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-podinfo\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.859164 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.859132    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-token\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.859471 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.859448    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fdgr\" (UniqueName: \"kubernetes.io/projected/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-kube-api-access-4fdgr\") pod \"e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") " pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:08.934370 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:08.934308    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:09.360520 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:09.360497    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"]
Nov 02 01:50:09.362674 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:50:09.362645    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1090bbf7_6cf8_43d7_b777_5bcaa301f76d.slice/crio-bd94eeb983dbd783544e1457030777930d6c3106e7bf0d48b6a6759958150b01 WatchSource:0}: Error finding container bd94eeb983dbd783544e1457030777930d6c3106e7bf0d48b6a6759958150b01: Status 404 returned error can't find the container with id bd94eeb983dbd783544e1457030777930d6c3106e7bf0d48b6a6759958150b01
Nov 02 01:50:09.364847 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:09.364814    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:50:09.364934 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:09.364876    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:50:09.364934 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:09.364902    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:50:09.979518 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:50:09.979483    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:50:10.354961 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:10.354871    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6" event={"ID":"1090bbf7-6cf8-43d7-b777-5bcaa301f76d","Type":"ContainerStarted","Data":"ec0f0a062766d2d608712fe1ee6612b18e4b10670cd118c78ddbe0f97f4f17ef"}
Nov 02 01:50:10.354961 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:10.354934    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6" event={"ID":"1090bbf7-6cf8-43d7-b777-5bcaa301f76d","Type":"ContainerStarted","Data":"bd94eeb983dbd783544e1457030777930d6c3106e7bf0d48b6a6759958150b01"}
Nov 02 01:50:10.532155 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:10.529349    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6" podStartSLOduration=2.529280628 podStartE2EDuration="2.529280628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:50:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:50:10.52868591 +0000 UTC m=+1099.741055720" watchObservedRunningTime="2026-11-02 01:50:10.529280628 +0000 UTC m=+1099.741650437"
Nov 02 01:50:10.939204 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:10.939169    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:10.940487 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:10.940153    4515 patch_prober.go:29] interesting pod/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 02 01:50:10.940487 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:10.940210    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6" podUID="1090bbf7-6cf8-43d7-b777-5bcaa301f76d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 02 01:50:11.944143 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:11.943530    4515 patch_prober.go:29] interesting pod/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 02 01:50:11.944143 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:11.943590    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6" podUID="1090bbf7-6cf8-43d7-b777-5bcaa301f76d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 02 01:50:12.938480 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:12.938444    4515 patch_prober.go:29] interesting pod/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 02 01:50:12.938753 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:12.938725    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6" podUID="1090bbf7-6cf8-43d7-b777-5bcaa301f76d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 02 01:50:13.937141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:13.935467    4515 patch_prober.go:29] interesting pod/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 02 01:50:13.937141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:13.935538    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6" podUID="1090bbf7-6cf8-43d7-b777-5bcaa301f76d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 02 01:50:14.943141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:14.937523    4515 patch_prober.go:29] interesting pod/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 02 01:50:14.943141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:14.937603    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6" podUID="1090bbf7-6cf8-43d7-b777-5bcaa301f76d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 02 01:50:16.296389 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:16.296352    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:16.296792 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:16.296730    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:16.297512 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:16.297486    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 01:50:24.918712 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:24.918675    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"]
Nov 02 01:50:24.922805 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:24.922780    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:24.980791 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:24.980759    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"]
Nov 02 01:50:24.992880 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:24.992850    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fmxt\" (UniqueName: \"kubernetes.io/projected/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-kube-api-access-8fmxt\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:24.993053 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:24.992956    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-dshm\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:24.993053 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:24.992985    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-home\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:24.993053 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:24.993017    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-model-cache\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:24.993237 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:24.993075    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-tmp-dir\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:24.993237 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:24.993142    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-tls-certs\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:25.027364 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:25.027166    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-ed0600\"/\"distinct-w4-ed0600-kserve-self-signed-certs\""
Nov 02 01:50:25.093833 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:25.093797    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fmxt\" (UniqueName: \"kubernetes.io/projected/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-kube-api-access-8fmxt\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:25.094032 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:25.093913    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-dshm\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:25.094032 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:25.093944    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-home\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:25.094032 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:25.093975    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-model-cache\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:25.094032 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:25.094000    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-tmp-dir\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:25.094032 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:25.094021    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-tls-certs\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:25.094466 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:25.094432    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-home\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:25.094527 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:25.094447    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-tmp-dir\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:25.094567 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:25.094520    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-model-cache\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:25.096360 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:25.096338    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-dshm\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:25.096539 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:25.096520    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-tls-certs\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:25.154607 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:25.154565    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fmxt\" (UniqueName: \"kubernetes.io/projected/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-kube-api-access-8fmxt\") pod \"distinct-w4-ed0600-kserve-699d4f9f98-mt4gg\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") " pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:25.234669 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:25.234636    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:25.633306 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:25.633267    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"]
Nov 02 01:50:25.635828 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:50:25.635801    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8e7a4cb_f925_40e2_9968_b8a0e19c283e.slice/crio-06a2c00e1e7fef34a1412b8d3047bdc8922e494c58a49ce6aeddb4e53fbe8df8 WatchSource:0}: Error finding container 06a2c00e1e7fef34a1412b8d3047bdc8922e494c58a49ce6aeddb4e53fbe8df8: Status 404 returned error can't find the container with id 06a2c00e1e7fef34a1412b8d3047bdc8922e494c58a49ce6aeddb4e53fbe8df8
Nov 02 01:50:26.442611 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:26.442575    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" event={"ID":"f8e7a4cb-f925-40e2-9968-b8a0e19c283e","Type":"ContainerStarted","Data":"3d3512e4037dec03455287ce170de81be3f473d14e2017ab1d9ac2eed8546756"}
Nov 02 01:50:26.442611 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:26.442618    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" event={"ID":"f8e7a4cb-f925-40e2-9968-b8a0e19c283e","Type":"ContainerStarted","Data":"06a2c00e1e7fef34a1412b8d3047bdc8922e494c58a49ce6aeddb4e53fbe8df8"}
Nov 02 01:50:26.559503 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:26.559445    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" podStartSLOduration=2.5594284050000002 podStartE2EDuration="2.559428405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:50:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:50:26.5588955 +0000 UTC m=+1115.771265309" watchObservedRunningTime="2026-11-02 01:50:26.559428405 +0000 UTC m=+1115.771798212"
Nov 02 01:50:27.235619 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:27.235585    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:35.235132 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:35.235014    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:35.248386 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:35.248360    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:35.494425 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:50:35.494348    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:50:39.959073 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:50:39.959030    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:51:01.926882 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:01.926846    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"]
Nov 02 01:51:01.930567 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:01.930549    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:01.997612 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:01.997576    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-model-cache\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:01.997775 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:01.997626    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-tmp-dir\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:01.997821 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:01.997779    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a471077b-3923-4355-8fda-135f0b8f038a-tls-certs\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:01.997857 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:01.997819    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-dshm\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:01.997945 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:01.997925    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnhbw\" (UniqueName: \"kubernetes.io/projected/a471077b-3923-4355-8fda-135f0b8f038a-kube-api-access-gnhbw\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:01.998032 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:01.998015    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-home\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:02.032307 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:02.032273    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-9470a3\"/\"distinct-w0-9470a3-kserve-self-signed-certs\""
Nov 02 01:51:02.034507 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:02.034485    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"]
Nov 02 01:51:02.099068 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:02.099030    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-home\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:02.099281 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:02.099080    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-model-cache\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:02.099281 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:02.099108    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-tmp-dir\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:02.099281 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:02.099232    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a471077b-3923-4355-8fda-135f0b8f038a-tls-certs\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:02.099281 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:02.099272    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-dshm\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:02.099493 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:02.099417    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnhbw\" (UniqueName: \"kubernetes.io/projected/a471077b-3923-4355-8fda-135f0b8f038a-kube-api-access-gnhbw\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:02.099574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:02.099554    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-model-cache\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:02.099615 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:02.099576    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-home\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:02.099651 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:02.099624    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-tmp-dir\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:02.101527 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:02.101508    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-dshm\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:02.101689 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:02.101672    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a471077b-3923-4355-8fda-135f0b8f038a-tls-certs\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:02.160780 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:02.160742    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnhbw\" (UniqueName: \"kubernetes.io/projected/a471077b-3923-4355-8fda-135f0b8f038a-kube-api-access-gnhbw\") pod \"distinct-w0-9470a3-kserve-99f6f6d55-bqbz5\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") " pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:02.242290 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:02.242253    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:02.648008 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:02.647982    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"]
Nov 02 01:51:02.650723 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:51:02.650697    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda471077b_3923_4355_8fda_135f0b8f038a.slice/crio-bbcf22e51adfb276d1a5aab78598337d574eb2c88c0e09daea327083c581dddb WatchSource:0}: Error finding container bbcf22e51adfb276d1a5aab78598337d574eb2c88c0e09daea327083c581dddb: Status 404 returned error can't find the container with id bbcf22e51adfb276d1a5aab78598337d574eb2c88c0e09daea327083c581dddb
Nov 02 01:51:03.622565 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:03.622529    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" event={"ID":"a471077b-3923-4355-8fda-135f0b8f038a","Type":"ContainerStarted","Data":"be8fcea67f04eb8cbf4829881dd5e80f2f80f5d42140be456b641df9b84638a3"}
Nov 02 01:51:03.622944 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:03.622573    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" event={"ID":"a471077b-3923-4355-8fda-135f0b8f038a","Type":"ContainerStarted","Data":"bbcf22e51adfb276d1a5aab78598337d574eb2c88c0e09daea327083c581dddb"}
Nov 02 01:51:03.739315 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:03.739263    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" podStartSLOduration=2.739249079 podStartE2EDuration="2.739249079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:51:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:51:03.738933552 +0000 UTC m=+1152.951303363" watchObservedRunningTime="2026-11-02 01:51:03.739249079 +0000 UTC m=+1152.951618894"
Nov 02 01:51:04.242692 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:04.242663    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:06.883721 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:06.883684    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"]
Nov 02 01:51:06.887392 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:06.887372    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:06.933637 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:06.933609    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"]
Nov 02 01:51:06.936977 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:06.936958    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:06.951224 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:06.951202    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istiod-ca-cert\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:06.951315 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:06.951238    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-workload-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:06.951315 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:06.951261    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-podinfo\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:06.951385 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:06.951361    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-data\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:06.951418 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:06.951402    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-token\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:06.951524 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:06.951500    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c95cv\" (UniqueName: \"kubernetes.io/projected/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-kube-api-access-c95cv\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:06.951582 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:06.951567    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-credential-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:06.951636 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:06.951624    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-workload-certs\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:06.951670 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:06.951644    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-envoy\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:06.975064 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:06.975035    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-ed0600-openshift-default-dockercfg-dhrkb\""
Nov 02 01:51:06.988970 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:06.988948    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"]
Nov 02 01:51:07.043373 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.043347    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-0ebdb3-openshift-default-dockercfg-hkbq9\""
Nov 02 01:51:07.053079 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053048    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-token\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.053265 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053168    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-token\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.053265 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053206    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c95cv\" (UniqueName: \"kubernetes.io/projected/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-kube-api-access-c95cv\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.053265 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053257    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-credential-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.053448 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053294    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-workload-certs\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.053448 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053328    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-envoy\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.053448 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053358    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-podinfo\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.053448 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053388    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-data\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.053448 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053441    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qql7b\" (UniqueName: \"kubernetes.io/projected/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-kube-api-access-qql7b\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.053728 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053489    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istiod-ca-cert\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.053728 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053528    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-workload-certs\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.053728 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053576    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-workload-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.053728 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053601    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-workload-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.053728 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053637    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-podinfo\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.053728 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053665    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istiod-ca-cert\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.053728 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053718    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-data\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.054056 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053750    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-credential-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.054056 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.053779    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-envoy\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.054971 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.054863    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-credential-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.055378 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.055350    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-workload-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.055479 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.055399    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-data\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.055583 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.055557    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istiod-ca-cert\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.055851 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.055792    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-workload-certs\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.058023 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.057998    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-podinfo\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.073615 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.073594    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-envoy\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.091829 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.091807    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"]
Nov 02 01:51:07.114931 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.114897    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-token\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.115029 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.114948    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c95cv\" (UniqueName: \"kubernetes.io/projected/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-kube-api-access-c95cv\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.155016 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.154956    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-workload-certs\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.155016 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.154992    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-workload-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.155016 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.155013    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istiod-ca-cert\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.155257 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.155152    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-credential-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.155257 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.155195    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-envoy\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.155359 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.155315    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-token\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.155359 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.155327    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-workload-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.155464 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.155420    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-podinfo\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.155464 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.155428    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-workload-certs\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.155464 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.155451    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-data\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.155615 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.155505    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qql7b\" (UniqueName: \"kubernetes.io/projected/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-kube-api-access-qql7b\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.155615 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.155526    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-credential-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.155724 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.155698    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istiod-ca-cert\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.155851 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.155832    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-data\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.157981 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.157959    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-envoy\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.158208 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.158188    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-podinfo\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.201789 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.201768    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:07.216197 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.216177    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-token\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.216609 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.216588    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qql7b\" (UniqueName: \"kubernetes.io/projected/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-kube-api-access-qql7b\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.249803 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.249770    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:07.632289 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.632252    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"]
Nov 02 01:51:07.672099 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:51:07.672071    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd2855c3_4dfa_4b84_a7b2_7bc7dd9b4004.slice/crio-c18b0af6b09a2191d5ef9b3fa3a276d8607c299f83a45d1d5473e3bbc104e8cf WatchSource:0}: Error finding container c18b0af6b09a2191d5ef9b3fa3a276d8607c299f83a45d1d5473e3bbc104e8cf: Status 404 returned error can't find the container with id c18b0af6b09a2191d5ef9b3fa3a276d8607c299f83a45d1d5473e3bbc104e8cf
Nov 02 01:51:07.674933 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.674899    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:51:07.675021 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.674970    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:51:07.675021 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.675000    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:51:07.685907 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.685881    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"]
Nov 02 01:51:07.689486 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:51:07.689462    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65acd2ae_98d9_4492_b3a0_a12f4d0fd8c9.slice/crio-c3bcdcfc45324cab9ed949d1c31825dece5ccebb0cbbb65ce6146581a2636fe5 WatchSource:0}: Error finding container c3bcdcfc45324cab9ed949d1c31825dece5ccebb0cbbb65ce6146581a2636fe5: Status 404 returned error can't find the container with id c3bcdcfc45324cab9ed949d1c31825dece5ccebb0cbbb65ce6146581a2636fe5
Nov 02 01:51:07.691909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.691875    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:51:07.691999 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.691947    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:51:07.691999 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:07.691991    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:51:08.681457 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:08.680695    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm" event={"ID":"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004","Type":"ContainerStarted","Data":"23090ea8c8193ff14c7651e63cddc3d9ce2bae01a5790b0263741c74e76a985d"}
Nov 02 01:51:08.682221 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:08.682200    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm" event={"ID":"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004","Type":"ContainerStarted","Data":"c18b0af6b09a2191d5ef9b3fa3a276d8607c299f83a45d1d5473e3bbc104e8cf"}
Nov 02 01:51:08.682304 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:08.682263    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf" event={"ID":"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9","Type":"ContainerStarted","Data":"9ef60c8f7803205461e1863dd91d08e9441f54b13468b9423a005c28bc6893b9"}
Nov 02 01:51:08.682927 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:08.682900    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf" event={"ID":"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9","Type":"ContainerStarted","Data":"c3bcdcfc45324cab9ed949d1c31825dece5ccebb0cbbb65ce6146581a2636fe5"}
Nov 02 01:51:08.901000 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:08.900946    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm" podStartSLOduration=2.900933164 podStartE2EDuration="2.900933164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:51:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:51:08.872618295 +0000 UTC m=+1158.084988104" watchObservedRunningTime="2026-11-02 01:51:08.900933164 +0000 UTC m=+1158.113302972"
Nov 02 01:51:09.004372 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:09.004321    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf" podStartSLOduration=3.004305835 podStartE2EDuration="3.004305835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:51:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:51:09.00267237 +0000 UTC m=+1158.215042179" watchObservedRunningTime="2026-11-02 01:51:09.004305835 +0000 UTC m=+1158.216675874"
Nov 02 01:51:09.202577 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:09.202534    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:09.203620 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:09.203594    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 02 01:51:09.203690 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:09.203670    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm" podUID="fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 02 01:51:09.261197 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:09.257475    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:09.265139 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:09.262264    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 02 01:51:09.265139 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:09.262355    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf" podUID="65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 02 01:51:09.996531 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:51:09.996491    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:51:10.202775 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:10.202747    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 02 01:51:10.202997 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:10.202977    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm" podUID="fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 02 01:51:10.253184 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:10.252535    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 02 01:51:10.253184 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:10.252607    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf" podUID="65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 02 01:51:11.202723 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:11.202685    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 02 01:51:11.203094 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:11.202736    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm" podUID="fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 02 01:51:11.251141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:11.250478    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 02 01:51:11.251141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:11.250538    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf" podUID="65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 02 01:51:12.202774 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:12.202730    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 02 01:51:12.203271 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:12.202785    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm" podUID="fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 02 01:51:12.250141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:12.249486    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:12.254140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:12.250874    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 02 01:51:12.254140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:12.250942    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf" podUID="65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 02 01:51:12.283920 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:12.283892    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:12.713455 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:12.713430    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:51:13.203088 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:13.203047    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 02 01:51:13.203547 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:13.203101    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm" podUID="fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 02 01:51:13.253610 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:13.253573    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 02 01:51:13.253905 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:13.253872    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf" podUID="65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 02 01:51:14.202474 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:14.202434    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 02 01:51:14.202670 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:14.202500    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm" podUID="fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 02 01:51:14.250406 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:14.250379    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 02 01:51:14.250761 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:14.250429    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf" podUID="65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 02 01:51:15.207436 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:15.207407    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:15.207687 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:15.207660    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:15.208359 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:15.208344    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 01:51:15.254946 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:15.254923    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:15.255344 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:15.255196    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:15.255788 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:15.255774    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 01:51:39.958278 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:51:39.958240    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:51:51.072631 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:51.072597    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 01:51:51.073198 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:51:51.072688    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 01:52:09.958526 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:52:09.958446    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:52:23.712516 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.712486    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"]
Nov 02 01:52:23.717341 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.717320    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.748999 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.748968    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-workload-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.749146 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.749003    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-istio-envoy\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.749146 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.749077    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-workload-certs\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.749146 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.749125    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be97e917-9b14-484e-90f1-b9ad1b678313-istio-token\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.749296 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.749224    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be97e917-9b14-484e-90f1-b9ad1b678313-istio-podinfo\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.749296 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.749256    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n52w2\" (UniqueName: \"kubernetes.io/projected/be97e917-9b14-484e-90f1-b9ad1b678313-kube-api-access-n52w2\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.749400 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.749295    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-credential-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.749454 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.749419    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-istio-data\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.749546 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.749526    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be97e917-9b14-484e-90f1-b9ad1b678313-istiod-ca-cert\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.850334 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.850294    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-workload-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.850506 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.850346    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-istio-envoy\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.850506 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.850387    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-workload-certs\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.850506 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.850421    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be97e917-9b14-484e-90f1-b9ad1b678313-istio-token\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.850506 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.850456    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be97e917-9b14-484e-90f1-b9ad1b678313-istio-podinfo\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.850506 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.850491    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n52w2\" (UniqueName: \"kubernetes.io/projected/be97e917-9b14-484e-90f1-b9ad1b678313-kube-api-access-n52w2\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.850786 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.850544    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-credential-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.850786 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.850613    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-istio-data\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.850786 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.850683    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be97e917-9b14-484e-90f1-b9ad1b678313-istiod-ca-cert\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.850786 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.850703    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-workload-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.851027 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.850806    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-workload-certs\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.851145 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.851108    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-credential-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.851230 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.851167    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-istio-data\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.851357 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.851330    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be97e917-9b14-484e-90f1-b9ad1b678313-istiod-ca-cert\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.852682 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.852661    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-istio-envoy\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.853470 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.853445    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be97e917-9b14-484e-90f1-b9ad1b678313-istio-podinfo\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.910827 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.910803    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be97e917-9b14-484e-90f1-b9ad1b678313-istio-token\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.910962 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.910852    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n52w2\" (UniqueName: \"kubernetes.io/projected/be97e917-9b14-484e-90f1-b9ad1b678313-kube-api-access-n52w2\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:23.923074 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:23.923053    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"]
Nov 02 01:52:24.030724 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:24.030652    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:24.466557 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:24.466533    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"]
Nov 02 01:52:24.468568 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:52:24.468544    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe97e917_9b14_484e_90f1_b9ad1b678313.slice/crio-fa77b55f479773fe99a2e59b9d946f54e5664a9e4f69ccaa989c6325cb6b1f4d WatchSource:0}: Error finding container fa77b55f479773fe99a2e59b9d946f54e5664a9e4f69ccaa989c6325cb6b1f4d: Status 404 returned error can't find the container with id fa77b55f479773fe99a2e59b9d946f54e5664a9e4f69ccaa989c6325cb6b1f4d
Nov 02 01:52:24.470912 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:24.470879    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:52:24.471021 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:24.470943    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:52:24.471021 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:24.470971    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:52:25.062003 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:25.061956    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8" event={"ID":"be97e917-9b14-484e-90f1-b9ad1b678313","Type":"ContainerStarted","Data":"168bd5a7c541f48cbc9c03191de137bff160a68be9ccf1aa0d5bcbdf2480f1a1"}
Nov 02 01:52:25.062507 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:25.062037    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8" event={"ID":"be97e917-9b14-484e-90f1-b9ad1b678313","Type":"ContainerStarted","Data":"fa77b55f479773fe99a2e59b9d946f54e5664a9e4f69ccaa989c6325cb6b1f4d"}
Nov 02 01:52:25.281080 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:25.281028    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8" podStartSLOduration=2.280997358 podStartE2EDuration="2.280997358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:52:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:52:25.230951134 +0000 UTC m=+1234.443320946" watchObservedRunningTime="2026-11-02 01:52:25.280997358 +0000 UTC m=+1234.493367242"
Nov 02 01:52:26.041140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:26.037175    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:26.049305 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:26.049269    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 02 01:52:26.049471 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:26.049338    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8" podUID="be97e917-9b14-484e-90f1-b9ad1b678313" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 02 01:52:27.040569 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:27.040528    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 02 01:52:27.041219 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:27.041186    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8" podUID="be97e917-9b14-484e-90f1-b9ad1b678313" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 02 01:52:28.035141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:28.033404    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 02 01:52:28.035141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:28.033476    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8" podUID="be97e917-9b14-484e-90f1-b9ad1b678313" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 02 01:52:29.032811 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:29.032420    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 02 01:52:29.032811 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:29.032475    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8" podUID="be97e917-9b14-484e-90f1-b9ad1b678313" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 02 01:52:30.032201 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:30.032164    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 02 01:52:30.032495 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:30.032466    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8" podUID="be97e917-9b14-484e-90f1-b9ad1b678313" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 02 01:52:31.031654 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:31.031612    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 02 01:52:31.032196 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:31.031674    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8" podUID="be97e917-9b14-484e-90f1-b9ad1b678313" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 02 01:52:32.056194 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:32.056151    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:32.056662 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:32.056491    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:32.057952 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:32.057925    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 01:52:39.588477 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.588389    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"]
Nov 02 01:52:39.592358 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.592336    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.693835 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.693798    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-55bc84\"/\"distinct-2-w5-55bc84-kserve-self-signed-certs\""
Nov 02 01:52:39.696125 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.696096    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"]
Nov 02 01:52:39.752865 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.752832    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-home\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.753037 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.752946    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-dshm\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.753037 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.752976    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/301ce5d3-728b-4edf-86d4-65a8ea792055-tls-certs\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.753137 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.753082    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-tmp-dir\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.753176 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.753142    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hx6h\" (UniqueName: \"kubernetes.io/projected/301ce5d3-728b-4edf-86d4-65a8ea792055-kube-api-access-8hx6h\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.753275 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.753252    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-model-cache\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.854358 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.854267    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-model-cache\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.854635 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.854606    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-home\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.854749 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.854675    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-model-cache\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.854749 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.854720    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-dshm\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.854749 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.854745    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/301ce5d3-728b-4edf-86d4-65a8ea792055-tls-certs\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.854924 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.854826    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-tmp-dir\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.854924 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.854866    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hx6h\" (UniqueName: \"kubernetes.io/projected/301ce5d3-728b-4edf-86d4-65a8ea792055-kube-api-access-8hx6h\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.855028 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.854982    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-home\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.855496 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.855472    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-tmp-dir\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.857956 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.857930    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/301ce5d3-728b-4edf-86d4-65a8ea792055-tls-certs\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.858538 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.858513    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-dshm\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.916308 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:39.916274    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hx6h\" (UniqueName: \"kubernetes.io/projected/301ce5d3-728b-4edf-86d4-65a8ea792055-kube-api-access-8hx6h\") pod \"distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") " pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:39.989455 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:52:39.989413    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:52:40.205752 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:40.205724    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:40.591809 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:40.591782    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"]
Nov 02 01:52:40.593824 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:52:40.593788    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod301ce5d3_728b_4edf_86d4_65a8ea792055.slice/crio-7bfa22a49ffc6707b41956eb8e2ea9e7d97a86ee7b177e0929cf3efd444c124c WatchSource:0}: Error finding container 7bfa22a49ffc6707b41956eb8e2ea9e7d97a86ee7b177e0929cf3efd444c124c: Status 404 returned error can't find the container with id 7bfa22a49ffc6707b41956eb8e2ea9e7d97a86ee7b177e0929cf3efd444c124c
Nov 02 01:52:41.143801 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:41.143766    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" event={"ID":"301ce5d3-728b-4edf-86d4-65a8ea792055","Type":"ContainerStarted","Data":"c7c6508e568ab597254acca3fdada287b377ff3df31e1a4a22e3c8884c14952c"}
Nov 02 01:52:41.143801 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:41.143811    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" event={"ID":"301ce5d3-728b-4edf-86d4-65a8ea792055","Type":"ContainerStarted","Data":"7bfa22a49ffc6707b41956eb8e2ea9e7d97a86ee7b177e0929cf3efd444c124c"}
Nov 02 01:52:41.206409 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:41.206343    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:41.262821 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:41.262765    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" podStartSLOduration=2.26274976 podStartE2EDuration="2.26274976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:52:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:52:41.260493448 +0000 UTC m=+1250.472863676" watchObservedRunningTime="2026-11-02 01:52:41.26274976 +0000 UTC m=+1250.475119568"
Nov 02 01:52:50.206541 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:50.206505    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:50.219609 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:50.219582    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:51.208353 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:51.208318    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:52:52.228283 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.228245    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"]
Nov 02 01:52:52.232052 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.232029    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.333479 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.333440    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-0ebdb3\"/\"openshift-service-ca.crt\""
Nov 02 01:52:52.333648 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.333477    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-0ebdb3\"/\"kube-root-ca.crt\""
Nov 02 01:52:52.333648 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.333522    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-0ebdb3\"/\"default-dockercfg-gtrzx\""
Nov 02 01:52:52.333648 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.333594    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-0ebdb3\"/\"distinct-2-w6-0ebdb3-kserve-self-signed-certs\""
Nov 02 01:52:52.336070 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.336051    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"]
Nov 02 01:52:52.397733 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.397703    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p4kw\" (UniqueName: \"kubernetes.io/projected/6357653d-8952-4aff-b01b-c1e3dc3969ce-kube-api-access-4p4kw\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.397915 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.397827    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6357653d-8952-4aff-b01b-c1e3dc3969ce-tls-certs\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.397986 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.397956    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-home\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.398059 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.397984    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-tmp-dir\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.398059 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.398021    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-model-cache\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.398197 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.398071    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-dshm\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.498932 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.498835    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6357653d-8952-4aff-b01b-c1e3dc3969ce-tls-certs\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.499112 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.498930    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-home\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.499112 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.498960    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-tmp-dir\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.499112 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.499079    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-model-cache\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.499319 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.499186    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-dshm\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.499374 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.499319    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p4kw\" (UniqueName: \"kubernetes.io/projected/6357653d-8952-4aff-b01b-c1e3dc3969ce-kube-api-access-4p4kw\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.499428 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.499381    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-home\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.499619 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.499600    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-model-cache\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.499713 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.499613    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-tmp-dir\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.501828 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.501800    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-dshm\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.501981 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.501962    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6357653d-8952-4aff-b01b-c1e3dc3969ce-tls-certs\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.560391 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.560351    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p4kw\" (UniqueName: \"kubernetes.io/projected/6357653d-8952-4aff-b01b-c1e3dc3969ce-kube-api-access-4p4kw\") pod \"distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") " pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:52.843546 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:52.843467    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:52:53.220036 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:53.220014    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"]
Nov 02 01:52:53.222162 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:52:53.222111    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6357653d_8952_4aff_b01b_c1e3dc3969ce.slice/crio-44120d471341ba81480d97d1e3cb96c209c519ca5aca47d95d54ddbabfb70c7d WatchSource:0}: Error finding container 44120d471341ba81480d97d1e3cb96c209c519ca5aca47d95d54ddbabfb70c7d: Status 404 returned error can't find the container with id 44120d471341ba81480d97d1e3cb96c209c519ca5aca47d95d54ddbabfb70c7d
Nov 02 01:52:54.214413 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:54.214383    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" event={"ID":"6357653d-8952-4aff-b01b-c1e3dc3969ce","Type":"ContainerStarted","Data":"865c13aa142fe81b1c8e3d35c4d8a9dfdafe976316e5fa55c47fcdb317272513"}
Nov 02 01:52:54.214868 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:54.214425    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" event={"ID":"6357653d-8952-4aff-b01b-c1e3dc3969ce","Type":"ContainerStarted","Data":"44120d471341ba81480d97d1e3cb96c209c519ca5aca47d95d54ddbabfb70c7d"}
Nov 02 01:52:54.332443 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:54.332390    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" podStartSLOduration=2.332376392 podStartE2EDuration="2.332376392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:52:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:52:54.330431525 +0000 UTC m=+1263.542801333" watchObservedRunningTime="2026-11-02 01:52:54.332376392 +0000 UTC m=+1263.544746218"
Nov 02 01:52:54.844156 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:52:54.844106    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:53:02.843662 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:02.843624    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:53:02.859240 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:02.859215    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:53:03.272298 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:03.272270    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:53:09.959006 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:53:09.958964    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:53:22.941277 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:22.941242    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"]
Nov 02 01:53:22.944773 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:22.944754    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:22.986266 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:22.986225    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-workload-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:22.986434 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:22.986388    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-credential-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:22.986527 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:22.986510    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-278fl\" (UniqueName: \"kubernetes.io/projected/07f6d265-73ef-481c-b583-73c648e7747c-kube-api-access-278fl\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:22.986572 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:22.986548    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f6d265-73ef-481c-b583-73c648e7747c-istio-token\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:22.986726 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:22.986706    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f6d265-73ef-481c-b583-73c648e7747c-istiod-ca-cert\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:22.986796 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:22.986778    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f6d265-73ef-481c-b583-73c648e7747c-istio-podinfo\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:22.986853 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:22.986820    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-workload-certs\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:22.986916 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:22.986898    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-istio-envoy\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:22.986958 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:22.986923    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-istio-data\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.088207 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.088174    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f6d265-73ef-481c-b583-73c648e7747c-istiod-ca-cert\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.088395 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.088226    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f6d265-73ef-481c-b583-73c648e7747c-istio-podinfo\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.088395 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.088260    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-workload-certs\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.088395 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.088294    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-istio-envoy\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.088395 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.088317    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-istio-data\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.088395 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.088350    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-workload-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.088631 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.088455    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-credential-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.088631 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.088541    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-278fl\" (UniqueName: \"kubernetes.io/projected/07f6d265-73ef-481c-b583-73c648e7747c-kube-api-access-278fl\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.088631 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.088572    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f6d265-73ef-481c-b583-73c648e7747c-istio-token\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.088786 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.088719    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-workload-certs\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.088846 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.088805    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-istio-data\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.088964 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.088876    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-workload-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.089008 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.088995    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f6d265-73ef-481c-b583-73c648e7747c-istiod-ca-cert\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.089043 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.089014    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-credential-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.090928 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.090897    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-istio-envoy\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.091036 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.091024    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f6d265-73ef-481c-b583-73c648e7747c-istio-podinfo\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.146715 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.146677    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"]
Nov 02 01:53:23.151645 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.151580    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f6d265-73ef-481c-b583-73c648e7747c-istio-token\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.152341 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.152315    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-278fl\" (UniqueName: \"kubernetes.io/projected/07f6d265-73ef-481c-b583-73c648e7747c-kube-api-access-278fl\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.260056 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.259962    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:23.694666 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.694641    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"]
Nov 02 01:53:23.697106 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:53:23.697081    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07f6d265_73ef_481c_b583_73c648e7747c.slice/crio-b82767556ea1a8f1eb1f5e6fed555bec13463e9106355ce30433166f68711418 WatchSource:0}: Error finding container b82767556ea1a8f1eb1f5e6fed555bec13463e9106355ce30433166f68711418: Status 404 returned error can't find the container with id b82767556ea1a8f1eb1f5e6fed555bec13463e9106355ce30433166f68711418
Nov 02 01:53:23.699377 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.699343    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:53:23.699438 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.699412    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:53:23.699473 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:23.699448    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:53:24.399195 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.399157    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv" event={"ID":"07f6d265-73ef-481c-b583-73c648e7747c","Type":"ContainerStarted","Data":"160d48e249e68bcffe3e820edd7ca58c4ef17a6ecb94d0b014e8168b78e74a15"}
Nov 02 01:53:24.399620 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.399217    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv" event={"ID":"07f6d265-73ef-481c-b583-73c648e7747c","Type":"ContainerStarted","Data":"b82767556ea1a8f1eb1f5e6fed555bec13463e9106355ce30433166f68711418"}
Nov 02 01:53:24.562422 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.562379    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"]
Nov 02 01:53:24.566158 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.566138    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.604017 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.603983    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-credential-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.604209 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.604031    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb1ff40c-c38c-474d-8522-6da91152739a-istio-podinfo\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.604209 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.604110    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-istio-envoy\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.604286 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.604220    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdp8r\" (UniqueName: \"kubernetes.io/projected/bb1ff40c-c38c-474d-8522-6da91152739a-kube-api-access-hdp8r\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.604362 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.604303    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-istio-data\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.604362 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.604347    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-workload-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.604428 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.604382    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-workload-certs\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.604474 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.604461    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb1ff40c-c38c-474d-8522-6da91152739a-istiod-ca-cert\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.604509 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.604484    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb1ff40c-c38c-474d-8522-6da91152739a-istio-token\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.610843 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.610793    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv" podStartSLOduration=2.610779533 podStartE2EDuration="2.610779533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:53:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:53:24.571187475 +0000 UTC m=+1293.783557283" watchObservedRunningTime="2026-11-02 01:53:24.610779533 +0000 UTC m=+1293.823149342"
Nov 02 01:53:24.615309 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.615284    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"]
Nov 02 01:53:24.618820 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.618804    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.648469 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.648437    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"]
Nov 02 01:53:24.705352 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705321    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-workload-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.705352 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705358    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb8fk\" (UniqueName: \"kubernetes.io/projected/90baf3f1-927b-44b1-b63b-3f6503b4fee3-kube-api-access-qb8fk\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.705585 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705389    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-workload-certs\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.705585 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705419    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istiod-ca-cert\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.705585 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705439    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-credential-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.705585 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705461    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-workload-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.705585 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705482    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb1ff40c-c38c-474d-8522-6da91152739a-istiod-ca-cert\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.705585 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705503    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb1ff40c-c38c-474d-8522-6da91152739a-istio-token\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.705585 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705537    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-podinfo\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.705585 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705570    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-token\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.705862 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705589    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-credential-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.705862 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705622    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb1ff40c-c38c-474d-8522-6da91152739a-istio-podinfo\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.705862 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705639    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-workload-certs\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.705862 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705666    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-istio-envoy\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.705862 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705714    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdp8r\" (UniqueName: \"kubernetes.io/projected/bb1ff40c-c38c-474d-8522-6da91152739a-kube-api-access-hdp8r\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.705862 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705742    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-envoy\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.705862 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705787    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-data\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.705862 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.705806    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-istio-data\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.706191 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.706169    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-istio-data\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.706464 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.706421    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-workload-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.706687 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.706668    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-workload-certs\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.707003 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.706982    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-credential-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.707100 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.707049    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb1ff40c-c38c-474d-8522-6da91152739a-istiod-ca-cert\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.708678 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.708646    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-istio-envoy\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.709087 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.709067    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb1ff40c-c38c-474d-8522-6da91152739a-istio-podinfo\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.762419 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.762379    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"]
Nov 02 01:53:24.771859 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.771824    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb1ff40c-c38c-474d-8522-6da91152739a-istio-token\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.772227 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.772181    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdp8r\" (UniqueName: \"kubernetes.io/projected/bb1ff40c-c38c-474d-8522-6da91152739a-kube-api-access-hdp8r\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.806741 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.806706    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-envoy\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.806922 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.806757    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-data\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.806922 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.806783    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb8fk\" (UniqueName: \"kubernetes.io/projected/90baf3f1-927b-44b1-b63b-3f6503b4fee3-kube-api-access-qb8fk\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.806922 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.806824    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istiod-ca-cert\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.806922 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.806842    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-credential-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.806922 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.806865    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-workload-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.806922 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.806923    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-podinfo\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.807285 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.806969    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-token\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.807285 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.807021    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-workload-certs\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.807285 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.807142    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-data\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.807521 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.807494    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istiod-ca-cert\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.807660 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.807599    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-workload-certs\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.807727 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.807653    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-credential-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.807727 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.807671    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-workload-socket\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.808994 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.808973    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-envoy\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.809504 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.809487    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-podinfo\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.867610 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.867557    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-token\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.871135 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.871095    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb8fk\" (UniqueName: \"kubernetes.io/projected/90baf3f1-927b-44b1-b63b-3f6503b4fee3-kube-api-access-qb8fk\") pod \"e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") " pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:24.879593 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.879562    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:24.957362 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:24.957341    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:25.260381 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:25.260297    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:25.261807 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:25.261778    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 02 01:53:25.261946 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:25.261823    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv" podUID="07f6d265-73ef-481c-b583-73c648e7747c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 02 01:53:25.352363 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:53:25.352326    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb1ff40c_c38c_474d_8522_6da91152739a.slice/crio-ea69d45602e503e3b817d71925cb9b93477dd53c43687935ba9a5ed5af5c787a WatchSource:0}: Error finding container ea69d45602e503e3b817d71925cb9b93477dd53c43687935ba9a5ed5af5c787a: Status 404 returned error can't find the container with id ea69d45602e503e3b817d71925cb9b93477dd53c43687935ba9a5ed5af5c787a
Nov 02 01:53:25.354310 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:25.354271    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"]
Nov 02 01:53:25.355524 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:25.355487    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:53:25.355644 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:25.355574    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:53:25.355644 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:25.355620    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:53:25.388212 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:25.388183    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"]
Nov 02 01:53:25.393373 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:53:25.393332    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90baf3f1_927b_44b1_b63b_3f6503b4fee3.slice/crio-e4e4024ec7065840f92b0a1525b82515527626acf26383c7060522328724b33a WatchSource:0}: Error finding container e4e4024ec7065840f92b0a1525b82515527626acf26383c7060522328724b33a: Status 404 returned error can't find the container with id e4e4024ec7065840f92b0a1525b82515527626acf26383c7060522328724b33a
Nov 02 01:53:25.396300 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:25.396258    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:53:25.396392 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:25.396352    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:53:25.396433 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:25.396398    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:53:25.412569 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:25.412535    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d" event={"ID":"bb1ff40c-c38c-474d-8522-6da91152739a","Type":"ContainerStarted","Data":"ea69d45602e503e3b817d71925cb9b93477dd53c43687935ba9a5ed5af5c787a"}
Nov 02 01:53:25.413862 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:25.413819    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4" event={"ID":"90baf3f1-927b-44b1-b63b-3f6503b4fee3","Type":"ContainerStarted","Data":"e4e4024ec7065840f92b0a1525b82515527626acf26383c7060522328724b33a"}
Nov 02 01:53:26.263141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:26.262273    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 02 01:53:26.263141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:26.262335    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv" podUID="07f6d265-73ef-481c-b583-73c648e7747c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 02 01:53:26.420777 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:26.420741    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d" event={"ID":"bb1ff40c-c38c-474d-8522-6da91152739a","Type":"ContainerStarted","Data":"e82d1666e38b91caae756825a1e29bdca7a8a6a45c9907bd39b9bd6bdee23b6e"}
Nov 02 01:53:26.422933 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:26.422898    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4" event={"ID":"90baf3f1-927b-44b1-b63b-3f6503b4fee3","Type":"ContainerStarted","Data":"dd411abacd787178ba930ac6ef5a43f602dd22456d7b8587607e7889d0f8f4cb"}
Nov 02 01:53:26.596204 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:26.595846    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d" podStartSLOduration=2.5958233379999998 podStartE2EDuration="2.595823338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:53:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:53:26.593626693 +0000 UTC m=+1295.805996503" watchObservedRunningTime="2026-11-02 01:53:26.595823338 +0000 UTC m=+1295.808193148"
Nov 02 01:53:26.899199 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:26.883309    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:26.899199 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:26.883454    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 02 01:53:26.899199 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:26.883498    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d" podUID="bb1ff40c-c38c-474d-8522-6da91152739a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 02 01:53:26.976142 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:26.962020    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 02 01:53:26.976142 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:26.962081    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4" podUID="90baf3f1-927b-44b1-b63b-3f6503b4fee3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 02 01:53:26.999140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:26.992938    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:27.274149 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:27.272328    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 02 01:53:27.274149 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:27.272403    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv" podUID="07f6d265-73ef-481c-b583-73c648e7747c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 02 01:53:27.885662 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:27.885618    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 02 01:53:27.886326 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:27.886293    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d" podUID="bb1ff40c-c38c-474d-8522-6da91152739a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 02 01:53:27.963718 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:27.963676    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 02 01:53:27.964031 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:27.963991    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4" podUID="90baf3f1-927b-44b1-b63b-3f6503b4fee3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 02 01:53:28.276143 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:28.274563    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 02 01:53:28.276143 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:28.274631    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv" podUID="07f6d265-73ef-481c-b583-73c648e7747c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 02 01:53:28.886142 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:28.885505    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 02 01:53:28.886142 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:28.885569    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d" podUID="bb1ff40c-c38c-474d-8522-6da91152739a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 02 01:53:28.965140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:28.964280    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 02 01:53:28.965140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:28.964347    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4" podUID="90baf3f1-927b-44b1-b63b-3f6503b4fee3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 02 01:53:29.275145 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:29.260900    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 02 01:53:29.275145 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:29.260962    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv" podUID="07f6d265-73ef-481c-b583-73c648e7747c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 02 01:53:29.884133 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:29.883304    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 02 01:53:29.884133 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:29.883703    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d" podUID="bb1ff40c-c38c-474d-8522-6da91152739a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 02 01:53:29.970617 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:29.970577    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 02 01:53:29.971255 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:29.971221    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4" podUID="90baf3f1-927b-44b1-b63b-3f6503b4fee3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 02 01:53:30.264660 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:30.264623    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 02 01:53:30.264941 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:30.264911    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv" podUID="07f6d265-73ef-481c-b583-73c648e7747c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 02 01:53:30.884172 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:30.883521    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 02 01:53:30.884172 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:30.883581    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d" podUID="bb1ff40c-c38c-474d-8522-6da91152739a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 02 01:53:30.962072 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:30.962036    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 02 01:53:30.962368 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:30.962339    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4" podUID="90baf3f1-927b-44b1-b63b-3f6503b4fee3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 02 01:53:31.690265 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:31.690226    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:31.691385 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:31.691358    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:31.692344 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:31.692324    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 01:53:31.880800 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:31.880760    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 02 01:53:31.880965 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:31.880826    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d" podUID="bb1ff40c-c38c-474d-8522-6da91152739a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 02 01:53:31.908768 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:31.908718    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4" podStartSLOduration=7.908705245 podStartE2EDuration="7.908705245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:53:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:53:26.74229564 +0000 UTC m=+1295.954665443" watchObservedRunningTime="2026-11-02 01:53:31.908705245 +0000 UTC m=+1301.121075052"
Nov 02 01:53:31.958652 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:31.958561    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 02 01:53:31.958652 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:31.958636    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4" podUID="90baf3f1-927b-44b1-b63b-3f6503b4fee3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 02 01:53:32.983535 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:32.983448    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:32.983992 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:32.983898    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:32.984803 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:32.984784    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 01:53:32.995276 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:32.995238    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:32.995722 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:32.995698    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:32.996402 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:32.996386    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 01:53:34.747464 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.747426    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"]
Nov 02 01:53:34.751603 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.751586    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.857853 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.857810    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-workload-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.858033 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.857953    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"]
Nov 02 01:53:34.858033 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.857966    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-workload-certs\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.858033 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.858020    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-istio-data\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.858205 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.858065    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4517f292-db6c-4474-9162-0d22eeda607a-istiod-ca-cert\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.858205 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.858173    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-647vl\" (UniqueName: \"kubernetes.io/projected/4517f292-db6c-4474-9162-0d22eeda607a-kube-api-access-647vl\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.858281 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.858226    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-istio-envoy\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.858281 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.858251    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4517f292-db6c-4474-9162-0d22eeda607a-istio-token\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.858349 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.858335    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-credential-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.858409 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.858393    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4517f292-db6c-4474-9162-0d22eeda607a-istio-podinfo\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.959626 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.959575    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-workload-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.959819 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.959680    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-workload-certs\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.959819 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.959715    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-istio-data\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.959819 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.959768    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4517f292-db6c-4474-9162-0d22eeda607a-istiod-ca-cert\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.959819 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.959806    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-647vl\" (UniqueName: \"kubernetes.io/projected/4517f292-db6c-4474-9162-0d22eeda607a-kube-api-access-647vl\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.960042 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.959853    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-istio-envoy\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.960042 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.959880    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4517f292-db6c-4474-9162-0d22eeda607a-istio-token\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.960042 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.959970    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-credential-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.960042 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.959984    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-workload-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.960301 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.960056    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4517f292-db6c-4474-9162-0d22eeda607a-istio-podinfo\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.960301 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.960223    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-workload-certs\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.960419 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.960383    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-credential-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.960565 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.960109    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-istio-data\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.960837 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.960583    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4517f292-db6c-4474-9162-0d22eeda607a-istiod-ca-cert\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.962944 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.962921    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-istio-envoy\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:34.963261 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:34.963240    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4517f292-db6c-4474-9162-0d22eeda607a-istio-podinfo\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:35.020600 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:35.020535    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4517f292-db6c-4474-9162-0d22eeda607a-istio-token\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:35.020770 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:35.020748    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-647vl\" (UniqueName: \"kubernetes.io/projected/4517f292-db6c-4474-9162-0d22eeda607a-kube-api-access-647vl\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:35.065099 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:35.065066    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:35.498846 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:35.498822    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"]
Nov 02 01:53:35.500833 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:53:35.500808    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4517f292_db6c_4474_9162_0d22eeda607a.slice/crio-686d10c36b0935f077bff1a845f24e854cdb9a7b5709a993b90b62787219f141 WatchSource:0}: Error finding container 686d10c36b0935f077bff1a845f24e854cdb9a7b5709a993b90b62787219f141: Status 404 returned error can't find the container with id 686d10c36b0935f077bff1a845f24e854cdb9a7b5709a993b90b62787219f141
Nov 02 01:53:35.503432 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:35.503391    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:53:35.503538 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:35.503470    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:53:35.503538 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:35.503500    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:53:35.597396 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:35.597350    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz" event={"ID":"4517f292-db6c-4474-9162-0d22eeda607a","Type":"ContainerStarted","Data":"00d4067d18816c9a8b9cd54df95472267324647c37d520e540aab223778ecf75"}
Nov 02 01:53:35.597571 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:35.597417    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz" event={"ID":"4517f292-db6c-4474-9162-0d22eeda607a","Type":"ContainerStarted","Data":"686d10c36b0935f077bff1a845f24e854cdb9a7b5709a993b90b62787219f141"}
Nov 02 01:53:35.719133 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:35.719066    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz" podStartSLOduration=1.719051184 podStartE2EDuration="1.719051184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:53:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:53:35.718188254 +0000 UTC m=+1304.930558061" watchObservedRunningTime="2026-11-02 01:53:35.719051184 +0000 UTC m=+1304.931420993"
Nov 02 01:53:36.065291 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:36.065255    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:36.066989 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:36.066959    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 02 01:53:36.067159 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:36.067012    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz" podUID="4517f292-db6c-4474-9162-0d22eeda607a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 02 01:53:37.065696 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:37.065664    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 02 01:53:37.066093 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:37.065713    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz" podUID="4517f292-db6c-4474-9162-0d22eeda607a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 02 01:53:38.065389 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.065357    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 02 01:53:38.065570 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.065408    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz" podUID="4517f292-db6c-4474-9162-0d22eeda607a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 02 01:53:38.116743 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.116712    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"]
Nov 02 01:53:38.120484 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.120465    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.201492 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.201455    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istiod-ca-cert\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.201667 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.201515    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-workload-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.201667 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.201638    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-data\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.201744 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.201713    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b6m9\" (UniqueName: \"kubernetes.io/projected/3f4a20c6-793d-44fd-988e-a1b950a71ea7-kube-api-access-4b6m9\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.201918 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.201903    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-podinfo\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.201967 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.201927    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-token\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.201967 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.201947    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-workload-certs\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.202030 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.201966    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-envoy\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.202084 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.202069    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-credential-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.224019 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.223981    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"]
Nov 02 01:53:38.303354 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.303317    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-workload-certs\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.303529 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.303352    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-envoy\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.303529 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.303459    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-credential-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.303529 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.303496    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istiod-ca-cert\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.303664 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.303541    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-workload-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.303664 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.303573    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-data\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.303664 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.303625    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b6m9\" (UniqueName: \"kubernetes.io/projected/3f4a20c6-793d-44fd-988e-a1b950a71ea7-kube-api-access-4b6m9\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.303830 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.303763    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-workload-certs\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.303893 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.303855    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-credential-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.303893 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.303884    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-podinfo\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.304003 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.303918    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-token\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.304082 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.304058    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-workload-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.304293 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.304273    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istiod-ca-cert\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.304380 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.304338    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-data\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.305708 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.305688    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-envoy\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.306467 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.306446    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-podinfo\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.365083 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.365004    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-token\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.365263 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.365139    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b6m9\" (UniqueName: \"kubernetes.io/projected/3f4a20c6-793d-44fd-988e-a1b950a71ea7-kube-api-access-4b6m9\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.459453 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.459414    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:38.892536 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.892507    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"]
Nov 02 01:53:38.894332 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:53:38.894305    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f4a20c6_793d_44fd_988e_a1b950a71ea7.slice/crio-4faa227cf1b68f41ad7bb0197f26c9e3a08dad05016e3e864b4b491639ccabfe WatchSource:0}: Error finding container 4faa227cf1b68f41ad7bb0197f26c9e3a08dad05016e3e864b4b491639ccabfe: Status 404 returned error can't find the container with id 4faa227cf1b68f41ad7bb0197f26c9e3a08dad05016e3e864b4b491639ccabfe
Nov 02 01:53:38.896768 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.896731    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:53:38.896869 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.896800    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:53:38.896869 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:38.896827    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:53:39.066440 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:39.066391    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 02 01:53:39.066623 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:39.066462    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz" podUID="4517f292-db6c-4474-9162-0d22eeda607a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 02 01:53:39.621639 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:39.621558    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x" event={"ID":"3f4a20c6-793d-44fd-988e-a1b950a71ea7","Type":"ContainerStarted","Data":"b88e57c88ba90343101c19464164375d9ef549fc58c1bd6d5f765a8f7c5bd9f4"}
Nov 02 01:53:39.621639 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:39.621632    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x" event={"ID":"3f4a20c6-793d-44fd-988e-a1b950a71ea7","Type":"ContainerStarted","Data":"4faa227cf1b68f41ad7bb0197f26c9e3a08dad05016e3e864b4b491639ccabfe"}
Nov 02 01:53:39.994488 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:53:39.994436    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:53:40.069137 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:40.068291    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 02 01:53:40.069137 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:40.068355    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz" podUID="4517f292-db6c-4474-9162-0d22eeda607a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 02 01:53:40.464139 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:40.460506    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:40.473399 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:40.473356    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 01:53:40.473601 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:40.473428    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x" podUID="3f4a20c6-793d-44fd-988e-a1b950a71ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 01:53:41.068458 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:41.068423    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 02 01:53:41.069056 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:41.069023    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz" podUID="4517f292-db6c-4474-9162-0d22eeda607a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 02 01:53:41.462140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:41.461573    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 01:53:41.462140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:41.461636    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x" podUID="3f4a20c6-793d-44fd-988e-a1b950a71ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 01:53:42.065767 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:42.065731    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 02 01:53:42.065988 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:42.065784    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz" podUID="4517f292-db6c-4474-9162-0d22eeda607a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 02 01:53:42.460634 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:42.460597    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 01:53:42.461028 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:42.460651    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x" podUID="3f4a20c6-793d-44fd-988e-a1b950a71ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 01:53:43.070512 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:43.070482    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:43.070878 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:43.070853    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:43.071619 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:43.071596    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 01:53:43.297742 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:43.292861    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x" podStartSLOduration=5.292841796 podStartE2EDuration="5.292841796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:53:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:53:39.7514842 +0000 UTC m=+1308.963854010" watchObservedRunningTime="2026-11-02 01:53:43.292841796 +0000 UTC m=+1312.505211606"
Nov 02 01:53:43.460919 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:43.460887    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 01:53:43.461340 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:43.460952    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x" podUID="3f4a20c6-793d-44fd-988e-a1b950a71ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 01:53:44.460243 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:44.460210    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 01:53:44.460398 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:44.460274    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x" podUID="3f4a20c6-793d-44fd-988e-a1b950a71ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 01:53:45.459950 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:45.459917    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 01:53:45.460361 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:45.459981    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x" podUID="3f4a20c6-793d-44fd-988e-a1b950a71ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 01:53:46.508636 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:46.508606    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:46.509560 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:46.509532    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:53:46.510782 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:53:46.510759    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 01:54:09.505175 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.505136    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"]
Nov 02 01:54:09.510666 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.510636    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.615473 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.615440    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"]
Nov 02 01:54:09.664435 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.664404    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-credential-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.664596 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.664442    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59aa703a-473b-4038-a9d8-6ae2e618f010-istiod-ca-cert\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.664656 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.664634    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-workload-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.664705 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.664692    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-data\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.664820 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.664799    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-workload-certs\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.664854 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.664828    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-envoy\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.664854 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.664844    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-podinfo\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.664918 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.664865    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-token\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.664918 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.664896    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4lhq\" (UniqueName: \"kubernetes.io/projected/59aa703a-473b-4038-a9d8-6ae2e618f010-kube-api-access-m4lhq\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.765964 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.765871    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-token\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.765964 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.765924    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4lhq\" (UniqueName: \"kubernetes.io/projected/59aa703a-473b-4038-a9d8-6ae2e618f010-kube-api-access-m4lhq\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.766210 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.765974    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-credential-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.766210 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.766007    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59aa703a-473b-4038-a9d8-6ae2e618f010-istiod-ca-cert\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.766319 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.766213    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-workload-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.766319 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.766283    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-data\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.766426 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.766380    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-workload-certs\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.766426 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.766407    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-envoy\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.766537 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.766431    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-podinfo\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.766671 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.766635    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-workload-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.766901 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.766875    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-credential-socket\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.766993 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.766917    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59aa703a-473b-4038-a9d8-6ae2e618f010-istiod-ca-cert\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.766993 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.766944    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-workload-certs\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.767132 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.767099    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-data\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.769336 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.769307    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-envoy\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.769630 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.769608    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-podinfo\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.827175 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.827144    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-token\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.827303 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:09.827260    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4lhq\" (UniqueName: \"kubernetes.io/projected/59aa703a-473b-4038-a9d8-6ae2e618f010-kube-api-access-m4lhq\") pod \"e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") " pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:09.958848 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:54:09.958810    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:54:10.123347 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:10.123263    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:10.552883 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:10.552857    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"]
Nov 02 01:54:10.554904 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:54:10.554870    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59aa703a_473b_4038_a9d8_6ae2e618f010.slice/crio-23ff6f4d7e1c735de01b544d00a05acad7b36531207c7e7df84c25870117985a WatchSource:0}: Error finding container 23ff6f4d7e1c735de01b544d00a05acad7b36531207c7e7df84c25870117985a: Status 404 returned error can't find the container with id 23ff6f4d7e1c735de01b544d00a05acad7b36531207c7e7df84c25870117985a
Nov 02 01:54:10.557405 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:10.557375    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:54:10.557470 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:10.557440    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:54:10.557507 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:10.557467    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:54:10.815699 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:10.815620    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn" event={"ID":"59aa703a-473b-4038-a9d8-6ae2e618f010","Type":"ContainerStarted","Data":"39a082441cda730a122f3d3933b915afe4bf61871bc6ce712417426c7f31bf4a"}
Nov 02 01:54:10.815699 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:10.815677    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn" event={"ID":"59aa703a-473b-4038-a9d8-6ae2e618f010","Type":"ContainerStarted","Data":"23ff6f4d7e1c735de01b544d00a05acad7b36531207c7e7df84c25870117985a"}
Nov 02 01:54:10.935666 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:10.935595    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn" podStartSLOduration=1.935577476 podStartE2EDuration="1.935577476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:54:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:54:10.934207522 +0000 UTC m=+1340.146577331" watchObservedRunningTime="2026-11-02 01:54:10.935577476 +0000 UTC m=+1340.147947284"
Nov 02 01:54:11.123938 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:11.123863    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:11.131311 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:11.131273    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 01:54:11.131591 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:11.131563    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn" podUID="59aa703a-473b-4038-a9d8-6ae2e618f010" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 01:54:12.124022 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:12.123988    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 01:54:12.124539 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:12.124044    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn" podUID="59aa703a-473b-4038-a9d8-6ae2e618f010" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 01:54:13.124149 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:13.124076    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 01:54:13.124648 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:13.124163    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn" podUID="59aa703a-473b-4038-a9d8-6ae2e618f010" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 01:54:14.124389 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:14.124355    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 01:54:14.124919 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:14.124892    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn" podUID="59aa703a-473b-4038-a9d8-6ae2e618f010" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 01:54:15.138263 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:15.138229    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 01:54:15.138841 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:15.138807    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn" podUID="59aa703a-473b-4038-a9d8-6ae2e618f010" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 01:54:16.123900 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:16.123866    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 01:54:16.124076 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:16.123928    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn" podUID="59aa703a-473b-4038-a9d8-6ae2e618f010" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 01:54:17.123724 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:17.123685    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 01:54:17.124158 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:17.123765    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn" podUID="59aa703a-473b-4038-a9d8-6ae2e618f010" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 01:54:18.128707 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:18.128677    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:18.129108 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:18.129086    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:18.129830 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:18.129812    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 01:54:21.993148 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:21.992219    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:22.024143 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:22.012218    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:22.073191 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:22.062213    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" podUID="747c597f-77cf-4287-a7a1-879565aec8ae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 02 01:54:22.087142 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:22.083228    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m" podUID="dab11213-84f0-4a27-bcd6-3f170312bba3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:22.158283 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:22.158228    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:22.203267 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:22.203210    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" podUID="301ce5d3-728b-4edf-86d4-65a8ea792055" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:22.203651 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:22.203620    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:22.266813 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:22.266704    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" podUID="6357653d-8952-4aff-b01b-c1e3dc3969ce" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:22.301737 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:22.301316    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Nov 02 01:54:22.486295 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:22.486222    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Nov 02 01:54:22.690761 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:22.690140    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc" podUID="dc4865ed-8a4f-4432-afc1-61d816b83090" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Nov 02 01:54:22.720134 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:22.719375    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" podUID="a471077b-3923-4355-8fda-135f0b8f038a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:22.873263 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:22.873205    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" podUID="228f299b-731d-4375-93a9-fa777801b0a1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Nov 02 01:54:23.003074 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.002219    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:23.006316 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.006289    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:54:23.017807 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.016201    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 02 01:54:23.017807 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.016610    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:54:23.074068 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.073394    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" podUID="747c597f-77cf-4287-a7a1-879565aec8ae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:23.074488 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.074323    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:54:23.161141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.136204    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m" podUID="dab11213-84f0-4a27-bcd6-3f170312bba3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:23.268230 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.233246    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:54:23.268230 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.209236    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:23.268230 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.209275    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" podUID="301ce5d3-728b-4edf-86d4-65a8ea792055" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Nov 02 01:54:23.268230 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.234891    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:54:23.268230 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.234944    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:54:23.268230 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.191062    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:23.268230 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.259227    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:54:23.279522 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.277308    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" podUID="6357653d-8952-4aff-b01b-c1e3dc3969ce" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded"
Nov 02 01:54:23.279522 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.277354    4515 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" podUID="a471077b-3923-4355-8fda-135f0b8f038a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:23.279522 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.277471    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:54:23.306355 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.306293    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:23.365143 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.340141    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:54:23.496295 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.496250    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Nov 02 01:54:23.497654 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.497631    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:54:23.558728 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.558636    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:54:23.607337 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.606931    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:54:23.619930 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.619902    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:54:23.620302 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.620284    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:54:23.652353 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.652322    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:54:23.658964 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.658526    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:54:23.712136 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.708018    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:54:23.748136 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.732799    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:54:23.762978 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.762949    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:54:23.789020 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:23.788995    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:54:29.637514 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:29.637478    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"]
Nov 02 01:54:29.637930 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:29.637709    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z" podUID="6cbffd1e-1b76-472b-b656-e2df763f243a" containerName="istio-proxy" containerID="cri-o://379293b0c90404c195e48a011478162f9f37b3496feea7e20fa5dba50605fe40" gracePeriod=30
Nov 02 01:54:34.727772 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.727741    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"]
Nov 02 01:54:34.731471 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.731455    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.834745 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.834580    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df6ef068-9dda-4168-b5ba-1768306ddb74-istiod-ca-cert\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.834745 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.834687    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-workload-certs\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.834966 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.834789    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-credential-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.834966 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.834823    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-data\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.834966 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.834876    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2qps\" (UniqueName: \"kubernetes.io/projected/df6ef068-9dda-4168-b5ba-1768306ddb74-kube-api-access-b2qps\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.834966 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.834939    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-envoy\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.835231 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.834985    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-workload-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.835231 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.835007    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-token\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.835231 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.835099    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-podinfo\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.836969 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.836943    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"]
Nov 02 01:54:34.924636 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.924609    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:54:34.936884 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.936857    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df6ef068-9dda-4168-b5ba-1768306ddb74-istiod-ca-cert\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.937034 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.936956    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-workload-certs\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.937034 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.937004    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-credential-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.937168 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.937036    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-data\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.937168 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.937099    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2qps\" (UniqueName: \"kubernetes.io/projected/df6ef068-9dda-4168-b5ba-1768306ddb74-kube-api-access-b2qps\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.937282 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.937188    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-envoy\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.937282 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.937250    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-workload-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.937282 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.937276    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-token\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.937428 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.937330    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-workload-certs\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.937428 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.937380    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-podinfo\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.937428 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.937392    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-data\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.937428 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.937413    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-credential-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.937640 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.937618    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-workload-socket\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.937920 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.937900    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df6ef068-9dda-4168-b5ba-1768306ddb74-istiod-ca-cert\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.939395 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.939372    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-envoy\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.939738 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.939715    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-podinfo\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.997535 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.997509    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-token\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:34.997741 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:34.997717    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2qps\" (UniqueName: \"kubernetes.io/projected/df6ef068-9dda-4168-b5ba-1768306ddb74-kube-api-access-b2qps\") pod \"e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") " pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:35.018448 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.018417    4515 generic.go:356] "Generic (PLEG): container finished" podID="6cbffd1e-1b76-472b-b656-e2df763f243a" containerID="379293b0c90404c195e48a011478162f9f37b3496feea7e20fa5dba50605fe40" exitCode=0
Nov 02 01:54:35.018599 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.018468    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z" event={"ID":"6cbffd1e-1b76-472b-b656-e2df763f243a","Type":"ContainerDied","Data":"379293b0c90404c195e48a011478162f9f37b3496feea7e20fa5dba50605fe40"}
Nov 02 01:54:35.018599 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.018476    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"
Nov 02 01:54:35.018599 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.018524    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z" event={"ID":"6cbffd1e-1b76-472b-b656-e2df763f243a","Type":"ContainerDied","Data":"3f93700b18cd9e3b6d51cf137cc482179f630853a573e1f93ed2250a682cb611"}
Nov 02 01:54:35.018599 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.018557    4515 scope.go:122] "RemoveContainer" containerID="379293b0c90404c195e48a011478162f9f37b3496feea7e20fa5dba50605fe40"
Nov 02 01:54:35.027510 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.027495    4515 scope.go:122] "RemoveContainer" containerID="379293b0c90404c195e48a011478162f9f37b3496feea7e20fa5dba50605fe40"
Nov 02 01:54:35.027772 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:54:35.027754    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"379293b0c90404c195e48a011478162f9f37b3496feea7e20fa5dba50605fe40\": container with ID starting with 379293b0c90404c195e48a011478162f9f37b3496feea7e20fa5dba50605fe40 not found: ID does not exist" containerID="379293b0c90404c195e48a011478162f9f37b3496feea7e20fa5dba50605fe40"
Nov 02 01:54:35.027815 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.027784    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"379293b0c90404c195e48a011478162f9f37b3496feea7e20fa5dba50605fe40"} err="failed to get container status \"379293b0c90404c195e48a011478162f9f37b3496feea7e20fa5dba50605fe40\": rpc error: code = NotFound desc = could not find container \"379293b0c90404c195e48a011478162f9f37b3496feea7e20fa5dba50605fe40\": container with ID starting with 379293b0c90404c195e48a011478162f9f37b3496feea7e20fa5dba50605fe40 not found: ID does not exist"
Nov 02 01:54:35.038008 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.037985    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-token\" (UniqueName: \"kubernetes.io/projected/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-token\") pod \"6cbffd1e-1b76-472b-b656-e2df763f243a\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") "
Nov 02 01:54:35.038104 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.038087    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-envoy\") pod \"6cbffd1e-1b76-472b-b656-e2df763f243a\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") "
Nov 02 01:54:35.038188 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.038142    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-credential-socket\") pod \"6cbffd1e-1b76-472b-b656-e2df763f243a\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") "
Nov 02 01:54:35.038242 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.038186    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6cbffd1e-1b76-472b-b656-e2df763f243a-kube-api-access-twxvt\" (UniqueName: \"kubernetes.io/projected/6cbffd1e-1b76-472b-b656-e2df763f243a-kube-api-access-twxvt\") pod \"6cbffd1e-1b76-472b-b656-e2df763f243a\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") "
Nov 02 01:54:35.038242 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.038231    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-workload-certs\") pod \"6cbffd1e-1b76-472b-b656-e2df763f243a\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") "
Nov 02 01:54:35.038343 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.038298    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-podinfo\") pod \"6cbffd1e-1b76-472b-b656-e2df763f243a\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") "
Nov 02 01:54:35.038444 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.038419    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-workload-socket\") pod \"6cbffd1e-1b76-472b-b656-e2df763f243a\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") "
Nov 02 01:54:35.038568 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.038455    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-data\") pod \"6cbffd1e-1b76-472b-b656-e2df763f243a\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") "
Nov 02 01:54:35.038568 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.038458    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-credential-socket" pod "6cbffd1e-1b76-472b-b656-e2df763f243a" (UID: "6cbffd1e-1b76-472b-b656-e2df763f243a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:54:35.038568 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.038484    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6cbffd1e-1b76-472b-b656-e2df763f243a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cbffd1e-1b76-472b-b656-e2df763f243a-istiod-ca-cert\") pod \"6cbffd1e-1b76-472b-b656-e2df763f243a\" (UID: \"6cbffd1e-1b76-472b-b656-e2df763f243a\") "
Nov 02 01:54:35.038568 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.038552    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-workload-certs" pod "6cbffd1e-1b76-472b-b656-e2df763f243a" (UID: "6cbffd1e-1b76-472b-b656-e2df763f243a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:54:35.038796 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.038704    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-workload-socket" pod "6cbffd1e-1b76-472b-b656-e2df763f243a" (UID: "6cbffd1e-1b76-472b-b656-e2df763f243a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:54:35.038796 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.038782    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-data" pod "6cbffd1e-1b76-472b-b656-e2df763f243a" (UID: "6cbffd1e-1b76-472b-b656-e2df763f243a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:54:35.038915 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.038793    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cbffd1e-1b76-472b-b656-e2df763f243a-istiod-ca-cert" pod "6cbffd1e-1b76-472b-b656-e2df763f243a" (UID: "6cbffd1e-1b76-472b-b656-e2df763f243a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 01:54:35.039366 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.039346    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:54:35.039471 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.039371    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:54:35.039471 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.039396    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:54:35.039471 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.039413    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:54:35.039471 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.039427    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cbffd1e-1b76-472b-b656-e2df763f243a-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:54:35.040279 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.040258    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-token" pod "6cbffd1e-1b76-472b-b656-e2df763f243a" (UID: "6cbffd1e-1b76-472b-b656-e2df763f243a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 01:54:35.040400 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.040338    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cbffd1e-1b76-472b-b656-e2df763f243a-kube-api-access-twxvt" pod "6cbffd1e-1b76-472b-b656-e2df763f243a" (UID: "6cbffd1e-1b76-472b-b656-e2df763f243a"). InnerVolumeSpecName "kube-api-access-twxvt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 01:54:35.040632 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.040605    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-envoy" pod "6cbffd1e-1b76-472b-b656-e2df763f243a" (UID: "6cbffd1e-1b76-472b-b656-e2df763f243a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 01:54:35.040733 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.040718    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-podinfo" pod "6cbffd1e-1b76-472b-b656-e2df763f243a" (UID: "6cbffd1e-1b76-472b-b656-e2df763f243a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 01:54:35.050223 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.050003    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:35.140698 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.140666    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twxvt\" (UniqueName: \"kubernetes.io/projected/6cbffd1e-1b76-472b-b656-e2df763f243a-kube-api-access-twxvt\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:54:35.140698 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.140697    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:54:35.140872 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.140708    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:54:35.140872 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.140718    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cbffd1e-1b76-472b-b656-e2df763f243a-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 01:54:35.471022 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.470988    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"]
Nov 02 01:54:35.495239 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.493789    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7w44z"]
Nov 02 01:54:35.519167 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.519133    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"]
Nov 02 01:54:35.523828 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:54:35.523801    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf6ef068_9dda_4168_b5ba_1768306ddb74.slice/crio-ddd99b89baf8e0d626ad6dffb1b6da610f2c8f91efe4f43f9090ebe8f0932691 WatchSource:0}: Error finding container ddd99b89baf8e0d626ad6dffb1b6da610f2c8f91efe4f43f9090ebe8f0932691: Status 404 returned error can't find the container with id ddd99b89baf8e0d626ad6dffb1b6da610f2c8f91efe4f43f9090ebe8f0932691
Nov 02 01:54:35.526235 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.526197    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:54:35.526340 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.526283    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:54:35.526340 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:35.526327    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:54:36.035140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:36.031068    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk" event={"ID":"df6ef068-9dda-4168-b5ba-1768306ddb74","Type":"ContainerStarted","Data":"2720db1774a79797afc107bf72b6a9127aeecd8a60a82b54858f74875e668734"}
Nov 02 01:54:36.035140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:36.031148    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk" event={"ID":"df6ef068-9dda-4168-b5ba-1768306ddb74","Type":"ContainerStarted","Data":"ddd99b89baf8e0d626ad6dffb1b6da610f2c8f91efe4f43f9090ebe8f0932691"}
Nov 02 01:54:36.053141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:36.051867    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:36.053141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:36.051999    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 01:54:36.053141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:36.052043    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk" podUID="df6ef068-9dda-4168-b5ba-1768306ddb74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 01:54:36.156486 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:36.156431    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk" podStartSLOduration=2.156417767 podStartE2EDuration="2.156417767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:54:36.154582079 +0000 UTC m=+1365.366951887" watchObservedRunningTime="2026-11-02 01:54:36.156417767 +0000 UTC m=+1365.368787582"
Nov 02 01:54:36.956609 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:36.956574    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6cbffd1e-1b76-472b-b656-e2df763f243a" path="/var/lib/kubelet/pods/6cbffd1e-1b76-472b-b656-e2df763f243a/volumes"
Nov 02 01:54:37.051435 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:37.051401    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 01:54:37.051898 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:37.051458    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk" podUID="df6ef068-9dda-4168-b5ba-1768306ddb74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 01:54:38.051143 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:38.050755    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 01:54:38.051143 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:38.050814    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk" podUID="df6ef068-9dda-4168-b5ba-1768306ddb74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 01:54:39.050712 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:39.050667    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 01:54:39.051156 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:39.050747    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk" podUID="df6ef068-9dda-4168-b5ba-1768306ddb74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 01:54:39.976336 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:54:39.976293    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:54:40.050470 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.050432    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 01:54:40.050672 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.050484    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk" podUID="df6ef068-9dda-4168-b5ba-1768306ddb74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 01:54:40.477607 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.477569    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"]
Nov 02 01:54:40.478130 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.478083    4515 cpu_manager.go:414] "containerMap: removing container" podUID="6cbffd1e-1b76-472b-b656-e2df763f243a" containerName="istio-proxy"
Nov 02 01:54:40.478130 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.478094    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6cbffd1e-1b76-472b-b656-e2df763f243a" containerName="istio-proxy"
Nov 02 01:54:40.478220 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.478216    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="6cbffd1e-1b76-472b-b656-e2df763f243a" containerName="istio-proxy"
Nov 02 01:54:40.481411 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.481394    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.595297 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.595259    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"]
Nov 02 01:54:40.607267 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.607071    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-envoy\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.607267 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.607145    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-token\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.607267 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.607319    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-workload-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.607759 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.607345    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-workload-certs\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.607759 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.607370    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98b67a95-2731-4fd4-83f5-f8a161e03a19-istiod-ca-cert\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.607759 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.607554    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxxgq\" (UniqueName: \"kubernetes.io/projected/98b67a95-2731-4fd4-83f5-f8a161e03a19-kube-api-access-jxxgq\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.607759 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.607612    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-podinfo\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.607759 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.607669    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-credential-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.607991 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.607848    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-data\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.709468 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.709427    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-workload-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.709468 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.709462    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-workload-certs\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.709725 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.709485    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98b67a95-2731-4fd4-83f5-f8a161e03a19-istiod-ca-cert\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.709725 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.709527    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxxgq\" (UniqueName: \"kubernetes.io/projected/98b67a95-2731-4fd4-83f5-f8a161e03a19-kube-api-access-jxxgq\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.709725 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.709549    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-podinfo\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.709725 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.709578    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-credential-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.709725 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.709669    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-data\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.709984 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.709742    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-envoy\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.709984 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.709786    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-token\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.709984 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.709892    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-workload-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.709984 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.709966    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-workload-certs\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.710247 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.710153    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-data\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.710438 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.710401    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-credential-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.710575 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.710558    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98b67a95-2731-4fd4-83f5-f8a161e03a19-istiod-ca-cert\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.712227 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.712203    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-podinfo\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.712410 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.712392    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-envoy\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.770701 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.770624    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-token\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.770841 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.770795    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxxgq\" (UniqueName: \"kubernetes.io/projected/98b67a95-2731-4fd4-83f5-f8a161e03a19-kube-api-access-jxxgq\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:40.795704 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:40.795659    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:41.050501 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:41.050419    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 01:54:41.050501 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:41.050473    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk" podUID="df6ef068-9dda-4168-b5ba-1768306ddb74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 01:54:41.257180 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:41.257144    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"]
Nov 02 01:54:41.295511 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:54:41.295467    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98b67a95_2731_4fd4_83f5_f8a161e03a19.slice/crio-0b0bed73ae93bc00fe1c8b6bd290e53692ac5c5544bc9ce8882d2ca81c71c70e WatchSource:0}: Error finding container 0b0bed73ae93bc00fe1c8b6bd290e53692ac5c5544bc9ce8882d2ca81c71c70e: Status 404 returned error can't find the container with id 0b0bed73ae93bc00fe1c8b6bd290e53692ac5c5544bc9ce8882d2ca81c71c70e
Nov 02 01:54:41.297963 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:41.297928    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:54:41.298040 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:41.298003    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:54:41.298040 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:41.298035    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:54:42.051559 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:42.051520    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 01:54:42.053327 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:42.053273    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk" podUID="df6ef068-9dda-4168-b5ba-1768306ddb74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 01:54:42.081356 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:42.081320    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx" event={"ID":"98b67a95-2731-4fd4-83f5-f8a161e03a19","Type":"ContainerStarted","Data":"7d22f257007f1dfd99c176c94d7c2b4f18cc7de73db0def2afe5e57382bc2c1e"}
Nov 02 01:54:42.081532 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:42.081377    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx" event={"ID":"98b67a95-2731-4fd4-83f5-f8a161e03a19","Type":"ContainerStarted","Data":"0b0bed73ae93bc00fe1c8b6bd290e53692ac5c5544bc9ce8882d2ca81c71c70e"}
Nov 02 01:54:42.796639 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:42.796597    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:42.798234 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:42.798201    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 01:54:42.798352 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:42.798264    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx" podUID="98b67a95-2731-4fd4-83f5-f8a161e03a19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 01:54:43.055150 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:43.054962    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:43.055628 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:43.055364    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:43.056157 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:43.056109    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 01:54:43.177560 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:43.177503    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx" podStartSLOduration=3.177488704 podStartE2EDuration="3.177488704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:54:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:54:42.23349231 +0000 UTC m=+1371.445862119" watchObservedRunningTime="2026-11-02 01:54:43.177488704 +0000 UTC m=+1372.389858515"
Nov 02 01:54:43.796548 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:43.796514    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 01:54:43.796762 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:43.796579    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx" podUID="98b67a95-2731-4fd4-83f5-f8a161e03a19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 01:54:44.796757 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:44.796728    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 01:54:44.797172 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:44.796783    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx" podUID="98b67a95-2731-4fd4-83f5-f8a161e03a19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 01:54:45.796589 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:45.796549    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 01:54:45.796764 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:45.796603    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx" podUID="98b67a95-2731-4fd4-83f5-f8a161e03a19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 01:54:46.796110 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:46.796074    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 01:54:46.796566 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:46.796158    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx" podUID="98b67a95-2731-4fd4-83f5-f8a161e03a19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 01:54:47.797012 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:47.796976    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 01:54:47.797521 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:47.797052    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx" podUID="98b67a95-2731-4fd4-83f5-f8a161e03a19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 01:54:48.800626 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:48.800596    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:48.801030 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:48.800856    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:48.801640 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:48.801625    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 01:54:57.566657 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:57.566600    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m" podUID="dab11213-84f0-4a27-bcd6-3f170312bba3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:57.666618 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:57.666568    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 02 01:54:57.669410 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:57.668213    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:57.693139 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:57.690192    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc" podUID="dc4865ed-8a4f-4432-afc1-61d816b83090" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Nov 02 01:54:57.693139 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:57.690767    4515 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m" podUID="dab11213-84f0-4a27-bcd6-3f170312bba3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/health\": context deadline exceeded"
Nov 02 01:54:57.717167 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:57.714210    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" podUID="747c597f-77cf-4287-a7a1-879565aec8ae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:57.717485 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:57.717453    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 02 01:54:57.744430 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:57.744364    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" podUID="6357653d-8952-4aff-b01b-c1e3dc3969ce" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:57.746224 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:57.746174    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:57.746514 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:57.746487    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" podUID="a471077b-3923-4355-8fda-135f0b8f038a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Nov 02 01:54:57.746671 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:57.746284    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" podUID="301ce5d3-728b-4edf-86d4-65a8ea792055" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:57.797284 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:57.797233    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:57.827585 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:57.827484    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Nov 02 01:54:57.861365 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:57.861319    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" podUID="228f299b-731d-4375-93a9-fa777801b0a1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Nov 02 01:54:58.008783 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.008734    4515 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" podUID="747c597f-77cf-4287-a7a1-879565aec8ae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:58.669425 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.669266    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:58.673753 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.673727    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:54:58.676111 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.676075    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:58.677139 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.676905    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:54:58.738142 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.715187    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" podUID="747c597f-77cf-4287-a7a1-879565aec8ae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 02 01:54:58.738142 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.716360    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:54:58.753316 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.741460    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc" podUID="dc4865ed-8a4f-4432-afc1-61d816b83090" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Nov 02 01:54:58.774181 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.743712    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:58.775690 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.746356    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" podUID="6357653d-8952-4aff-b01b-c1e3dc3969ce" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:58.776641 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.747348    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:58.776864 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.747388    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" podUID="a471077b-3923-4355-8fda-135f0b8f038a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:58.780168 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.747647    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" podUID="301ce5d3-728b-4edf-86d4-65a8ea792055" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:58.780168 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.770813    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:54:58.780168 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.777533    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:54:58.780168 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.777576    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:54:58.780168 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.777610    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:54:58.780168 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.777644    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:54:58.780168 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.777677    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:54:58.818608 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.800664    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:58.818608 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.801061    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:54:58.844643 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.844599    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Nov 02 01:54:58.844968 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.844944    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:54:58.874153 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.864314    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" podUID="228f299b-731d-4375-93a9-fa777801b0a1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:58.874754 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.874713    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:54:59.010819 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:58.998627    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:54:59.148251 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:59.148211    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 01:54:59.152135 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:59.148477    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz" podUID="4517f292-db6c-4474-9162-0d22eeda607a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:59.159720 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:59.159598    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 01:54:59.159952 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:59.159924    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk" podUID="df6ef068-9dda-4168-b5ba-1768306ddb74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:59.646252 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:59.646200    4515 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/health\": context deadline exceeded"
Nov 02 01:54:59.693254 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:59.693202    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 02 01:54:59.779141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:59.697192    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Nov 02 01:54:59.779141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:59.741208    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" podUID="747c597f-77cf-4287-a7a1-879565aec8ae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:59.779141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:59.760879    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc" podUID="dc4865ed-8a4f-4432-afc1-61d816b83090" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:59.779141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:59.776282    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 02 01:54:59.779141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:59.776699    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" podUID="6357653d-8952-4aff-b01b-c1e3dc3969ce" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:59.812142 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:59.796067    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" podUID="a471077b-3923-4355-8fda-135f0b8f038a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:59.817385 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:59.796215    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" podUID="301ce5d3-728b-4edf-86d4-65a8ea792055" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:59.817795 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:59.812426    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:54:59.910764 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:59.910661    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" podUID="228f299b-731d-4375-93a9-fa777801b0a1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Nov 02 01:54:59.978139 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:54:59.977190    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:00.137394 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.137347    4515 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:00.336290 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.336243    4515 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc" podUID="dc4865ed-8a4f-4432-afc1-61d816b83090" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/health\": context deadline exceeded"
Nov 02 01:55:00.609313 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.609197    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m" podUID="dab11213-84f0-4a27-bcd6-3f170312bba3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": context deadline exceeded"
Nov 02 01:55:00.680519 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.680483    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 01:55:00.680788 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.680759    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np" podUID="47069c5d-070f-425b-ba94-a69d0585fd29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:00.815298 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.812276    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" podUID="6357653d-8952-4aff-b01b-c1e3dc3969ce" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:00.815298 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.812665    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 02 01:55:00.815298 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.812954    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Nov 02 01:55:00.815298 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.813258    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" podUID="747c597f-77cf-4287-a7a1-879565aec8ae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 02 01:55:00.815298 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.813685    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 02 01:55:00.847138 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.838879    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:00.847138 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.839255    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc" podUID="dc4865ed-8a4f-4432-afc1-61d816b83090" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Nov 02 01:55:00.849297 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.848412    4515 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/health\": context deadline exceeded"
Nov 02 01:55:00.849297 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.848496    4515 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/health\": context deadline exceeded"
Nov 02 01:55:00.849297 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.848531    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" podUID="a471077b-3923-4355-8fda-135f0b8f038a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:00.853139 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.823244    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" podUID="301ce5d3-728b-4edf-86d4-65a8ea792055" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:00.978423 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.952930    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" podUID="228f299b-731d-4375-93a9-fa777801b0a1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Nov 02 01:55:00.979964 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.979205    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:00.980433 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:00.979258    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:01.218289 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.218247    4515 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" podUID="301ce5d3-728b-4edf-86d4-65a8ea792055" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": context deadline exceeded"
Nov 02 01:55:01.659141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.657473    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m" podUID="dab11213-84f0-4a27-bcd6-3f170312bba3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:01.814143 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.813638    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" podUID="747c597f-77cf-4287-a7a1-879565aec8ae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:01.814143 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.813844    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" podUID="6357653d-8952-4aff-b01b-c1e3dc3969ce" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded"
Nov 02 01:55:01.814143 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.814034    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 02 01:55:01.856995 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.856945    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" podUID="a471077b-3923-4355-8fda-135f0b8f038a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:01.943027 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.869460    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 02 01:55:01.969610 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.872556    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 02 01:55:01.969610 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.872908    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc" podUID="dc4865ed-8a4f-4432-afc1-61d816b83090" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Nov 02 01:55:01.969610 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.886273    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" podUID="301ce5d3-728b-4edf-86d4-65a8ea792055" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:01.969610 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.894435    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:55:01.969610 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.932426    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Nov 02 01:55:01.969610 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.942503    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 01:55:01.969610 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.958847    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx" podUID="444174f7-6dd7-411d-b625-e3edd0f214ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:01.980065 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.980016    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" podUID="228f299b-731d-4375-93a9-fa777801b0a1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Nov 02 01:55:01.980747 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.980328    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:01.981314 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.980926    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:01.981854 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:01.981836    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:55:02.241196 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:02.215308    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:55:02.315311 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:02.315274    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:55:02.338354 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:02.338325    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:55:02.729317 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:02.729271    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m" podUID="dab11213-84f0-4a27-bcd6-3f170312bba3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:02.783142 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:02.749689    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:55:02.798026 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:02.797995    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:55:02.930657 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:02.930449    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 02 01:55:02.938804 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:02.936383    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" podUID="6357653d-8952-4aff-b01b-c1e3dc3969ce" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded"
Nov 02 01:55:02.987628 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:02.978947    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:02.987628 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:02.978972    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" podUID="301ce5d3-728b-4edf-86d4-65a8ea792055" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Nov 02 01:55:02.987628 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:02.980227    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 02 01:55:02.987628 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:02.980796    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" podUID="228f299b-731d-4375-93a9-fa777801b0a1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Nov 02 01:55:02.987628 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:02.981873    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Nov 02 01:55:02.987628 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:02.982824    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Nov 02 01:55:03.002097 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:02.997396    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:55:03.031461 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:03.031430    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:55:03.046169 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:03.045347    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:55:03.136399 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:03.134260    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:55:03.148132 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:03.145870    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:55:03.149929 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:03.149163    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:55:03.163136 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:03.159408    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:55:03.169035 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:03.169006    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:55:09.961506 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:55:09.961463    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:55:19.721161 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.721125    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"]
Nov 02 01:55:19.725005 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.724988    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.777406 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.777371    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 02 01:55:19.777406 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.777371    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 02 01:55:19.777625 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.777420    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 02 01:55:19.826137 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.819888    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8pkj\" (UniqueName: \"kubernetes.io/projected/2f990ce6-351c-4a76-87b9-308d20c1b828-kube-api-access-g8pkj\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.826137 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.819999    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f990ce6-351c-4a76-87b9-308d20c1b828-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.826137 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.820020    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.826137 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.820084    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.826137 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.820175    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.826137 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.820254    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.826137 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.820291    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.826964 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.826933    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-f8tcq\""
Nov 02 01:55:19.831625 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.831600    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"]
Nov 02 01:55:19.921195 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.921158    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f990ce6-351c-4a76-87b9-308d20c1b828-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.921195 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.921199    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.921429 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.921266    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.921594 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.921568    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.921880 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.921854    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.922013 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.921917    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.922013 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.921956    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8pkj\" (UniqueName: \"kubernetes.io/projected/2f990ce6-351c-4a76-87b9-308d20c1b828-kube-api-access-g8pkj\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.922158 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.922093    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.923712 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.923690    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f990ce6-351c-4a76-87b9-308d20c1b828-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.923905 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.923887    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.924593 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.924571    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.924661 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.924646    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.982952 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.982881    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:19.982952 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:19.982895    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8pkj\" (UniqueName: \"kubernetes.io/projected/2f990ce6-351c-4a76-87b9-308d20c1b828-kube-api-access-g8pkj\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zwstb\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:20.037999 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:20.037970    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:20.470590 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:20.470557    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"]
Nov 02 01:55:20.472373 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:55:20.472346    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f990ce6_351c_4a76_87b9_308d20c1b828.slice/crio-cdb1401dfb8242b8093f1e53c07444c981eb08445cd2c8f61ab4909eb9093b92 WatchSource:0}: Error finding container cdb1401dfb8242b8093f1e53c07444c981eb08445cd2c8f61ab4909eb9093b92: Status 404 returned error can't find the container with id cdb1401dfb8242b8093f1e53c07444c981eb08445cd2c8f61ab4909eb9093b92
Nov 02 01:55:20.474773 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:20.474740    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:55:20.474865 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:20.474799    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:55:21.092592 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:21.092549    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb" event={"ID":"2f990ce6-351c-4a76-87b9-308d20c1b828","Type":"ContainerStarted","Data":"6c19547249532be0904f59eb0bf824ce93c62053ccda32b2347e643167aef52b"}
Nov 02 01:55:21.093009 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:21.092615    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb" event={"ID":"2f990ce6-351c-4a76-87b9-308d20c1b828","Type":"ContainerStarted","Data":"cdb1401dfb8242b8093f1e53c07444c981eb08445cd2c8f61ab4909eb9093b92"}
Nov 02 01:55:21.093009 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:21.092641    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:21.094872 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:21.094841    4515 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-zwstb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 01:55:21.095020 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:21.094911    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb" podUID="2f990ce6-351c-4a76-87b9-308d20c1b828" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 01:55:21.212101 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:21.212036    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb" podStartSLOduration=2.212019541 podStartE2EDuration="2.212019541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:55:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:55:21.211581729 +0000 UTC m=+1410.423951539" watchObservedRunningTime="2026-11-02 01:55:21.212019541 +0000 UTC m=+1410.424389351"
Nov 02 01:55:22.099460 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:22.099425    4515 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-zwstb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 01:55:22.099959 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:22.099486    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb" podUID="2f990ce6-351c-4a76-87b9-308d20c1b828" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 01:55:23.894851 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:23.894818    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"]
Nov 02 01:55:23.898608 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:23.898591    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:23.980591 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:23.980547    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-podinfo\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:23.980800 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:23.980603    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-workload-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:23.980800 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:23.980716    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-envoy\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:23.980941 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:23.980806    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k64s\" (UniqueName: \"kubernetes.io/projected/90e69d4e-ddd4-4ec5-b334-40f7a882731b-kube-api-access-5k64s\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:23.980941 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:23.980843    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istiod-ca-cert\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:23.981056 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:23.981028    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-data\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:23.981151 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:23.981132    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-credential-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:23.981238 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:23.981166    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-token\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:23.981238 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:23.981201    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-workload-certs\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.003016 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.002983    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"]
Nov 02 01:55:24.007095 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.007072    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"]
Nov 02 01:55:24.007265 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.007179    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.083689 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.083426    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istiod-ca-cert\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.083689 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.083497    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-podinfo\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.083689 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.083521    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-podinfo\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.083689 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.083549    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-workload-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.083689 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.083577    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-credential-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.083689 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.083627    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-envoy\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.083689 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.083662    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-envoy\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.084282 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.083938    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k64s\" (UniqueName: \"kubernetes.io/projected/90e69d4e-ddd4-4ec5-b334-40f7a882731b-kube-api-access-5k64s\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.084282 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.083985    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"]
Nov 02 01:55:24.084282 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.084155    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-workload-certs\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.084627 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.084601    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8nhs\" (UniqueName: \"kubernetes.io/projected/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-kube-api-access-v8nhs\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.084778 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.084669    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istiod-ca-cert\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.084778 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.084742    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-workload-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.084778 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.084750    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-workload-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.085004 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.084792    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-token\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.085004 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.084860    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-data\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.085004 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.084926    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-data\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.085269 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.085020    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-credential-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.085269 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.085050    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-token\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.085269 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.085084    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-workload-certs\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.085605 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.085555    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-data\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.085605 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.085577    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-credential-socket\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.085798 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.085733    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-workload-certs\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.085857 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.085805    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istiod-ca-cert\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.089151 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.087210    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-envoy\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.089151 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.088243    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-podinfo\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.145640 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.145561    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-token\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.145795 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.145772    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k64s\" (UniqueName: \"kubernetes.io/projected/90e69d4e-ddd4-4ec5-b334-40f7a882731b-kube-api-access-5k64s\") pod \"e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") " pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.186199 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.186112    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istiod-ca-cert\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.186394 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.186218    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-podinfo\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.186394 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.186244    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-credential-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.186394 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.186286    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-envoy\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.186394 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.186340    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-workload-certs\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.186394 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.186360    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8nhs\" (UniqueName: \"kubernetes.io/projected/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-kube-api-access-v8nhs\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.186672 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.186435    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-workload-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.186672 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.186470    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-token\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.186672 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.186531    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-data\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.186830 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.186665    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-credential-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.186830 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.186737    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-workload-certs\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.186938 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.186902    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-data\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.187005 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.186973    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istiod-ca-cert\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.187005 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.186978    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-workload-socket\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.188641 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.188614    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-envoy\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.188918 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.188899    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-podinfo\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.211803 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.211770    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:24.247462 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.247432    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-token\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.247616 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.247519    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8nhs\" (UniqueName: \"kubernetes.io/projected/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-kube-api-access-v8nhs\") pod \"e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") " pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.321575 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.321538    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:24.646562 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.646529    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"]
Nov 02 01:55:24.648061 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:55:24.648034    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90e69d4e_ddd4_4ec5_b334_40f7a882731b.slice/crio-303fb418eab6a33a2b75eec99ee6c16aba9f92b0c14fba244a2573d576c09b4b WatchSource:0}: Error finding container 303fb418eab6a33a2b75eec99ee6c16aba9f92b0c14fba244a2573d576c09b4b: Status 404 returned error can't find the container with id 303fb418eab6a33a2b75eec99ee6c16aba9f92b0c14fba244a2573d576c09b4b
Nov 02 01:55:24.650626 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.650589    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:55:24.650738 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.650672    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:55:24.650738 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.650716    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:55:24.706477 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.706453    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"]
Nov 02 01:55:24.707717 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:55:24.707687    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a2f9447_a086_4541_8a70_4e6dc5b98ac0.slice/crio-ea3680026f45bdbe8b13766d50d489d1a4b875c45544c1942ddd07a23a3be9f5 WatchSource:0}: Error finding container ea3680026f45bdbe8b13766d50d489d1a4b875c45544c1942ddd07a23a3be9f5: Status 404 returned error can't find the container with id ea3680026f45bdbe8b13766d50d489d1a4b875c45544c1942ddd07a23a3be9f5
Nov 02 01:55:24.710761 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.710722    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:55:24.710830 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.710814    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:55:24.710884 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:24.710868    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:55:25.100079 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.100040    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 01:55:25.122224 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.121833    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw" event={"ID":"3a2f9447-a086-4541-8a70-4e6dc5b98ac0","Type":"ContainerStarted","Data":"673c3195b20a388e0613bfb62afcd0ba68c6994082fdefebbcf3a8acdf101068"}
Nov 02 01:55:25.122224 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.121896    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw" event={"ID":"3a2f9447-a086-4541-8a70-4e6dc5b98ac0","Type":"ContainerStarted","Data":"ea3680026f45bdbe8b13766d50d489d1a4b875c45544c1942ddd07a23a3be9f5"}
Nov 02 01:55:25.123909 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.123875    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq" event={"ID":"90e69d4e-ddd4-4ec5-b334-40f7a882731b","Type":"ContainerStarted","Data":"f7e91945dd61064976deb30e7076fa929dbe1322555b13ae73f029267fe720bc"}
Nov 02 01:55:25.124053 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.123922    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq" event={"ID":"90e69d4e-ddd4-4ec5-b334-40f7a882731b","Type":"ContainerStarted","Data":"303fb418eab6a33a2b75eec99ee6c16aba9f92b0c14fba244a2573d576c09b4b"}
Nov 02 01:55:25.212216 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.212097    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:25.227287 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.227254    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 02 01:55:25.227476 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.227317    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq" podUID="90e69d4e-ddd4-4ec5-b334-40f7a882731b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 02 01:55:25.322641 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.322608    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:25.324399 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.324373    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 02 01:55:25.324509 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.324418    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw" podUID="3a2f9447-a086-4541-8a70-4e6dc5b98ac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 02 01:55:25.347279 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.347070    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq" podStartSLOduration=2.34705227 podStartE2EDuration="2.34705227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:55:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:55:25.346585255 +0000 UTC m=+1414.558955094" watchObservedRunningTime="2026-11-02 01:55:25.34705227 +0000 UTC m=+1414.559422079"
Nov 02 01:55:25.467271 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.467194    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw" podStartSLOduration=2.467173153 podStartE2EDuration="2.467173153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:55:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:55:25.466732158 +0000 UTC m=+1414.679101966" watchObservedRunningTime="2026-11-02 01:55:25.467173153 +0000 UTC m=+1414.679542964"
Nov 02 01:55:25.867192 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.867051    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"]
Nov 02 01:55:25.884479 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.883659    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:25.912199 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.910755    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b05cc018-433c-4f3a-9708-95e3bdde50ff-istiod-ca-cert\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:25.912199 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.910804    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7qvd\" (UniqueName: \"kubernetes.io/projected/b05cc018-433c-4f3a-9708-95e3bdde50ff-kube-api-access-b7qvd\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:25.912199 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.910837    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-workload-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:25.912199 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.910899    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-podinfo\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:25.912199 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.910930    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-envoy\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:25.912199 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.910966    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-data\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:25.912199 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.911018    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-credential-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:25.912199 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.911194    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-token\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:25.912199 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.911264    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-workload-certs\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:25.988369 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:25.988332    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"]
Nov 02 01:55:26.012710 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.012677    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b05cc018-433c-4f3a-9708-95e3bdde50ff-istiod-ca-cert\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.012919 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.012715    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7qvd\" (UniqueName: \"kubernetes.io/projected/b05cc018-433c-4f3a-9708-95e3bdde50ff-kube-api-access-b7qvd\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.012919 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.012747    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-workload-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.012919 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.012783    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-podinfo\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.012919 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.012801    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-envoy\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.012919 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.012824    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-data\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.012919 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.012863    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-credential-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.013751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.012948    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-token\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.013751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.013035    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-workload-certs\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.014751 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.014721    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-workload-certs\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.015406 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.015376    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-data\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.015519 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.015426    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-credential-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.016008 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.015983    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b05cc018-433c-4f3a-9708-95e3bdde50ff-istiod-ca-cert\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.016917 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.016891    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-workload-socket\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.018874 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.018825    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-envoy\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.020311 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.020292    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-podinfo\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.083979 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.083941    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-token\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.086192 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.086146    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7qvd\" (UniqueName: \"kubernetes.io/projected/b05cc018-433c-4f3a-9708-95e3bdde50ff-kube-api-access-b7qvd\") pod \"e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") " pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.200084 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.200045    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:26.213729 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.213686    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 02 01:55:26.213815 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.213762    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq" podUID="90e69d4e-ddd4-4ec5-b334-40f7a882731b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 02 01:55:26.347140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.324104    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 02 01:55:26.347140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.324188    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw" podUID="3a2f9447-a086-4541-8a70-4e6dc5b98ac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 02 01:55:26.679142 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.679045    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"]
Nov 02 01:55:26.712044 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:55:26.712012    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb05cc018_433c_4f3a_9708_95e3bdde50ff.slice/crio-e26b80e055255eced077a109b6957e34c5cd89c4718b03e3a073c48d00e3c33f WatchSource:0}: Error finding container e26b80e055255eced077a109b6957e34c5cd89c4718b03e3a073c48d00e3c33f: Status 404 returned error can't find the container with id e26b80e055255eced077a109b6957e34c5cd89c4718b03e3a073c48d00e3c33f
Nov 02 01:55:26.714718 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.714683    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:55:26.714800 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.714752    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:55:26.714800 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:26.714782    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:55:27.190733 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:27.190298    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2" event={"ID":"b05cc018-433c-4f3a-9708-95e3bdde50ff","Type":"ContainerStarted","Data":"c826576cbc49668c03a27ae04712cffdcd28a52c74ea8affcfd33a1793e62c35"}
Nov 02 01:55:27.190733 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:27.190461    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2" event={"ID":"b05cc018-433c-4f3a-9708-95e3bdde50ff","Type":"ContainerStarted","Data":"e26b80e055255eced077a109b6957e34c5cd89c4718b03e3a073c48d00e3c33f"}
Nov 02 01:55:27.200967 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:27.200918    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:27.202574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:27.202334    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 01:55:27.202574 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:27.202378    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2" podUID="b05cc018-433c-4f3a-9708-95e3bdde50ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 01:55:27.212145 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:27.212072    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 02 01:55:27.212242 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:27.212153    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq" podUID="90e69d4e-ddd4-4ec5-b334-40f7a882731b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 02 01:55:27.322037 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:27.322003    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 02 01:55:27.322196 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:27.322066    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw" podUID="3a2f9447-a086-4541-8a70-4e6dc5b98ac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 02 01:55:27.386199 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:27.376281    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2" podStartSLOduration=2.37625673 podStartE2EDuration="2.37625673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:55:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:55:27.339724328 +0000 UTC m=+1416.552094133" watchObservedRunningTime="2026-11-02 01:55:27.37625673 +0000 UTC m=+1416.588626539"
Nov 02 01:55:28.201142 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:28.200564    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 01:55:28.201142 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:28.200628    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2" podUID="b05cc018-433c-4f3a-9708-95e3bdde50ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 01:55:28.213342 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:28.213302    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 02 01:55:28.213534 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:28.213380    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq" podUID="90e69d4e-ddd4-4ec5-b334-40f7a882731b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 02 01:55:28.322261 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:28.322228    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 02 01:55:28.322418 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:28.322280    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw" podUID="3a2f9447-a086-4541-8a70-4e6dc5b98ac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 02 01:55:29.204141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:29.203228    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 01:55:29.204141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:29.203290    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2" podUID="b05cc018-433c-4f3a-9708-95e3bdde50ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 01:55:29.212091 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:29.212062    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 02 01:55:29.212248 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:29.212135    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq" podUID="90e69d4e-ddd4-4ec5-b334-40f7a882731b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 02 01:55:29.322421 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:29.322385    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 02 01:55:29.322676 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:29.322649    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw" podUID="3a2f9447-a086-4541-8a70-4e6dc5b98ac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 02 01:55:30.201534 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:30.201496    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 01:55:30.201816 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:30.201786    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2" podUID="b05cc018-433c-4f3a-9708-95e3bdde50ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 01:55:30.213695 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:30.213574    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 02 01:55:30.213695 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:30.213630    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq" podUID="90e69d4e-ddd4-4ec5-b334-40f7a882731b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 02 01:55:30.333406 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:30.333274    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 02 01:55:30.333406 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:30.333342    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw" podUID="3a2f9447-a086-4541-8a70-4e6dc5b98ac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 02 01:55:30.784142 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:30.770211    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Nov 02 01:55:30.846144 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:30.845149    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 02 01:55:30.861150 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:30.857247    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" podUID="a471077b-3923-4355-8fda-135f0b8f038a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:31.093352 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.093249    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" podUID="747c597f-77cf-4287-a7a1-879565aec8ae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 02 01:55:31.112473 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.103335    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m" podUID="dab11213-84f0-4a27-bcd6-3f170312bba3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": context deadline exceeded"
Nov 02 01:55:31.142140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.131070    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:31.142624 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.131108    4515 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/health\": context deadline exceeded"
Nov 02 01:55:31.142747 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.131147    4515 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/health\": context deadline exceeded"
Nov 02 01:55:31.142836 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.131167    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:31.162140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.131187    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc" podUID="dc4865ed-8a4f-4432-afc1-61d816b83090" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Nov 02 01:55:31.291598 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.154538    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:31.291598 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.155776    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Nov 02 01:55:31.291598 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.218824    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 02 01:55:31.291598 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.219491    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 01:55:31.291598 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.259713    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq" podUID="90e69d4e-ddd4-4ec5-b334-40f7a882731b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 02 01:55:31.291598 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.259748    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2" podUID="b05cc018-433c-4f3a-9708-95e3bdde50ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 01:55:31.331688 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.331644    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 02 01:55:31.380416 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.380317    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw" podUID="3a2f9447-a086-4541-8a70-4e6dc5b98ac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 02 01:55:31.782139 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.778676    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:31.782139 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.780098    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:55:31.845360 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.845295    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" podUID="6357653d-8952-4aff-b01b-c1e3dc3969ce" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:31.849620 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.849581    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" podUID="301ce5d3-728b-4edf-86d4-65a8ea792055" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:31.851020 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.850985    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 02 01:55:31.852283 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.852263    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:55:31.857442 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.857409    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" podUID="a471077b-3923-4355-8fda-135f0b8f038a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:31.858943 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:31.858921    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:55:32.195200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.156918    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 02 01:55:32.195200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.165330    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" podUID="747c597f-77cf-4287-a7a1-879565aec8ae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 02 01:55:32.195200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.166503    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m" podUID="dab11213-84f0-4a27-bcd6-3f170312bba3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": context deadline exceeded"
Nov 02 01:55:32.195200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.167550    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 02 01:55:32.195200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.168554    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:55:32.195200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.168613    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:55:32.195200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.168647    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:55:32.195200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.168680    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:55:32.364140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.333777    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" podUID="228f299b-731d-4375-93a9-fa777801b0a1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Nov 02 01:55:32.364140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.359423    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:32.380208 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.370942    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:55:32.544990 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.522990    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 01:55:32.681958 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.528571    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 02 01:55:32.682338 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.539699    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Nov 02 01:55:32.682608 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.539534    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc" podUID="dc4865ed-8a4f-4432-afc1-61d816b83090" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:32.682799 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.594343    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 02 01:55:32.770139 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.682297    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw" podUID="3a2f9447-a086-4541-8a70-4e6dc5b98ac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 02 01:55:32.770139 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.682927    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq" podUID="90e69d4e-ddd4-4ec5-b334-40f7a882731b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 02 01:55:32.801279 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.799301    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2" podUID="b05cc018-433c-4f3a-9708-95e3bdde50ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 01:55:32.801279 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:32.799747    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:55:33.030142 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.008228    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:55:33.127069 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.062051    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:33.155474 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.137605    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" podUID="6357653d-8952-4aff-b01b-c1e3dc3969ce" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:33.155474 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.138191    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:33.155474 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.138439    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" podUID="301ce5d3-728b-4edf-86d4-65a8ea792055" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Nov 02 01:55:33.155474 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.138689    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" podUID="a471077b-3923-4355-8fda-135f0b8f038a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:33.188398 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.171301    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:33.188398 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.171673    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 02 01:55:33.188398 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.171921    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" podUID="747c597f-77cf-4287-a7a1-879565aec8ae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 02 01:55:33.204465 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.201278    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 01:55:33.204465 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.201334    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2" podUID="b05cc018-433c-4f3a-9708-95e3bdde50ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 01:55:33.228336 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.228282    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 02 01:55:33.228634 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.228584    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq" podUID="90e69d4e-ddd4-4ec5-b334-40f7a882731b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 02 01:55:33.287213 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.242228    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m" podUID="dab11213-84f0-4a27-bcd6-3f170312bba3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:33.334259 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.326475    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 02 01:55:33.334259 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.326535    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw" podUID="3a2f9447-a086-4541-8a70-4e6dc5b98ac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 02 01:55:33.352145 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.335560    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" podUID="228f299b-731d-4375-93a9-fa777801b0a1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:33.354166 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.354137    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:55:33.355553 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.355536    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:55:33.356094 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.356080    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:55:33.387293 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.384701    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:33.531215 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.529486    4515 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" podUID="a471077b-3923-4355-8fda-135f0b8f038a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:33.698203 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.686190    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Nov 02 01:55:33.805147 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:33.800422    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc" podUID="dc4865ed-8a4f-4432-afc1-61d816b83090" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:34.100309 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.100209    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:34.154352 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.154285    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" podUID="a471077b-3923-4355-8fda-135f0b8f038a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:34.156230 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.156179    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" podUID="6357653d-8952-4aff-b01b-c1e3dc3969ce" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded"
Nov 02 01:55:34.159036 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.158996    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 02 01:55:34.164380 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.164329    4515 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" podUID="6357653d-8952-4aff-b01b-c1e3dc3969ce" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": context deadline exceeded"
Nov 02 01:55:34.165214 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.165173    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" podUID="301ce5d3-728b-4edf-86d4-65a8ea792055" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Nov 02 01:55:34.173510 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.173463    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" podUID="747c597f-77cf-4287-a7a1-879565aec8ae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 02 01:55:34.191161 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.181863    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 02 01:55:34.191161 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.190427    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 02 01:55:34.212480 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.212433    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 01:55:34.212787 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.212740    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2" podUID="b05cc018-433c-4f3a-9708-95e3bdde50ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 01:55:34.213966 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.213942    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 02 01:55:34.214179 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.214148    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq" podUID="90e69d4e-ddd4-4ec5-b334-40f7a882731b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 02 01:55:34.256200 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.243980    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m" podUID="dab11213-84f0-4a27-bcd6-3f170312bba3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:34.342141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.338760    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 01:55:34.356395 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.356285    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" podUID="228f299b-731d-4375-93a9-fa777801b0a1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Nov 02 01:55:34.369308 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.369264    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 02 01:55:34.369555 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.369525    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw" podUID="3a2f9447-a086-4541-8a70-4e6dc5b98ac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 02 01:55:34.414679 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.414612    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:34.560934 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.560823    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 01:55:34.674243 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.672274    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 01:55:34.723260 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.723219    4515 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" podUID="228f299b-731d-4375-93a9-fa777801b0a1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:34.826141 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.825613    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc" podUID="dc4865ed-8a4f-4432-afc1-61d816b83090" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:34.963350 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:34.960660    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 01:55:35.201859 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.197425    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:35.215072 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.214957    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 01:55:35.215285 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.215046    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2" podUID="b05cc018-433c-4f3a-9708-95e3bdde50ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 01:55:35.335482 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.335435    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" podUID="a471077b-3923-4355-8fda-135f0b8f038a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Nov 02 01:55:35.379363 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.379325    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 01:55:35.383649 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.383618    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 01:55:35.384251 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.384232    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m"
Nov 02 01:55:35.426139 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.415752    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 02 01:55:35.426749 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.418475    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Nov 02 01:55:35.426919 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.418627    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" podUID="228f299b-731d-4375-93a9-fa777801b0a1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:35.520758 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.520669    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 01:55:35.632426 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.631643    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 01:55:35.640676 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.638333    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 01:55:35.640676 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.639632    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 01:55:35.684781 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.684646    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:35.820988 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.815545    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 01:55:35.860184 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.851110    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 01:55:35.924498 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.923527    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:35.924745 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.924731    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:35.928019 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:35.927665    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 01:55:36.200715 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:36.200671    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 01:55:36.200952 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:36.200775    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2" podUID="b05cc018-433c-4f3a-9708-95e3bdde50ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 01:55:36.448628 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:36.448565    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 01:55:36.448999 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:36.448700    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw" podUID="3a2f9447-a086-4541-8a70-4e6dc5b98ac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:37.360839 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:37.360805    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:37.361311 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:37.361286    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:37.362307 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:37.362282    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 01:55:38.242303 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:38.242266    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 01:55:38.242764 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:38.242334    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2" podUID="b05cc018-433c-4f3a-9708-95e3bdde50ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 01:55:38.454217 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:38.454184    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:38.454593 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:38.454570    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:38.455519 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:55:38.455498    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 01:55:39.965365 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:55:39.965316    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:56:09.960018 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:56:09.959985    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:56:39.958753 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:56:39.958716    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:56:51.111873 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:56:51.111842    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 01:56:51.113700 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:56:51.113675    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 01:57:09.964404 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:57:09.964362    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:57:31.152477 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.152438    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"]
Nov 02 01:57:31.156450 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.156432    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.202010 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.201971    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"]
Nov 02 01:57:31.205718 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.205699    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.259041 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.259000    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"]
Nov 02 01:57:31.316037 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.316001    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"]
Nov 02 01:57:31.341756 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.341719    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18653351-22cb-40db-ae44-1982a38f0a4c-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.341925 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.341768    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18653351-22cb-40db-ae44-1982a38f0a4c-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.341925 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.341795    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18653351-22cb-40db-ae44-1982a38f0a4c-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.341925 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.341910    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.342049 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.342028    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.342156 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.342137    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.342209 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.342174    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.342261 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.342216    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.342261 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.342246    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18653351-22cb-40db-ae44-1982a38f0a4c-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.342335 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.342281    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.342399 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.342376    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnffw\" (UniqueName: \"kubernetes.io/projected/dbce4552-66b5-4a8f-ad14-edacdc87ea52-kube-api-access-nnffw\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.342456 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.342440    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.342515 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.342500    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18653351-22cb-40db-ae44-1982a38f0a4c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.342557 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.342532    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18653351-22cb-40db-ae44-1982a38f0a4c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.342675 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.342661    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.342715 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.342706    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18653351-22cb-40db-ae44-1982a38f0a4c-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.342845 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.342831    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgrj6\" (UniqueName: \"kubernetes.io/projected/18653351-22cb-40db-ae44-1982a38f0a4c-kube-api-access-xgrj6\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.342893 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.342870    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18653351-22cb-40db-ae44-1982a38f0a4c-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.445316 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.445281    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnffw\" (UniqueName: \"kubernetes.io/projected/dbce4552-66b5-4a8f-ad14-edacdc87ea52-kube-api-access-nnffw\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.445533 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.445337    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.445533 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.445373    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18653351-22cb-40db-ae44-1982a38f0a4c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.445533 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.445409    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18653351-22cb-40db-ae44-1982a38f0a4c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.445533 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.445519    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.445762 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.445575    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18653351-22cb-40db-ae44-1982a38f0a4c-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.445762 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.445635    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgrj6\" (UniqueName: \"kubernetes.io/projected/18653351-22cb-40db-ae44-1982a38f0a4c-kube-api-access-xgrj6\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.445762 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.445684    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18653351-22cb-40db-ae44-1982a38f0a4c-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.445762 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.445741    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18653351-22cb-40db-ae44-1982a38f0a4c-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.445959 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.445793    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18653351-22cb-40db-ae44-1982a38f0a4c-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.445959 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.445843    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18653351-22cb-40db-ae44-1982a38f0a4c-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.445959 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.445892    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.446105 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.445988    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.446105 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.446097    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.446233 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.446160    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.446233 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.446205    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.446337 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.446244    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18653351-22cb-40db-ae44-1982a38f0a4c-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.446337 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.446294    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.447243 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.447216    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.447385 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.447359    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18653351-22cb-40db-ae44-1982a38f0a4c-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.448188 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.448167    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.448325 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.448301    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18653351-22cb-40db-ae44-1982a38f0a4c-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.448990 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.448967    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18653351-22cb-40db-ae44-1982a38f0a4c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.452755 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.452703    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.452979 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.452957    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.453058 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.452983    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.453314 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.453294    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18653351-22cb-40db-ae44-1982a38f0a4c-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.453986 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.453962    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18653351-22cb-40db-ae44-1982a38f0a4c-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.454145 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.454022    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.454523 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.454497    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18653351-22cb-40db-ae44-1982a38f0a4c-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.456020 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.456000    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18653351-22cb-40db-ae44-1982a38f0a4c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.457301 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.457271    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.508128 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.508087    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnffw\" (UniqueName: \"kubernetes.io/projected/dbce4552-66b5-4a8f-ad14-edacdc87ea52-kube-api-access-nnffw\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.513416 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.513387    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18653351-22cb-40db-ae44-1982a38f0a4c-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.513416 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.513396    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xs4xg\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.513879 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.513860    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgrj6\" (UniqueName: \"kubernetes.io/projected/18653351-22cb-40db-ae44-1982a38f0a4c-kube-api-access-xgrj6\") pod \"maas-default-gateway-openshift-default-778b7b5c47-v8gx8\" (UID: \"18653351-22cb-40db-ae44-1982a38f0a4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.518790 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.518769    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:31.770681 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.770594    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:31.953911 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.953881    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"]
Nov 02 01:57:31.955057 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:57:31.955029    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbce4552_66b5_4a8f_ad14_edacdc87ea52.slice/crio-0a19c313d942d1d73a3edd1fd848a0e588cc385a7048091873a64a2b93c87bb9 WatchSource:0}: Error finding container 0a19c313d942d1d73a3edd1fd848a0e588cc385a7048091873a64a2b93c87bb9: Status 404 returned error can't find the container with id 0a19c313d942d1d73a3edd1fd848a0e588cc385a7048091873a64a2b93c87bb9
Nov 02 01:57:31.957502 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.957471    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:57:31.957570 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.957535    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:57:31.957570 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:31.957566    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:57:32.152994 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:32.152931    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"]
Nov 02 01:57:32.156336 ip-10-0-138-75 kubenswrapper[4515]: W1102 01:57:32.156303    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18653351_22cb_40db_ae44_1982a38f0a4c.slice/crio-5e47f03d6179282f98121d3afeed9745a1c75b8be7ab281431a2ae1e17365275 WatchSource:0}: Error finding container 5e47f03d6179282f98121d3afeed9745a1c75b8be7ab281431a2ae1e17365275: Status 404 returned error can't find the container with id 5e47f03d6179282f98121d3afeed9745a1c75b8be7ab281431a2ae1e17365275
Nov 02 01:57:32.159514 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:32.159474    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:57:32.159609 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:32.159550    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:57:32.159609 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:32.159580    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 01:57:32.299941 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:32.299897    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg" event={"ID":"dbce4552-66b5-4a8f-ad14-edacdc87ea52","Type":"ContainerStarted","Data":"a78258ff2753e157bc6fcf46d270506c66560411ef315e5ecb5e6d335aaf5ab4"}
Nov 02 01:57:32.300150 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:32.299972    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg" event={"ID":"dbce4552-66b5-4a8f-ad14-edacdc87ea52","Type":"ContainerStarted","Data":"0a19c313d942d1d73a3edd1fd848a0e588cc385a7048091873a64a2b93c87bb9"}
Nov 02 01:57:32.303146 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:32.302187    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8" event={"ID":"18653351-22cb-40db-ae44-1982a38f0a4c","Type":"ContainerStarted","Data":"c0e6a760a2e4defc56aceeeeb056e32219599460768c48100df885cbbf48108b"}
Nov 02 01:57:32.303146 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:32.302235    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8" event={"ID":"18653351-22cb-40db-ae44-1982a38f0a4c","Type":"ContainerStarted","Data":"5e47f03d6179282f98121d3afeed9745a1c75b8be7ab281431a2ae1e17365275"}
Nov 02 01:57:32.421857 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:32.421749    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg" podStartSLOduration=1.421727029 podStartE2EDuration="1.421727029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:57:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:57:32.419019865 +0000 UTC m=+1541.631389673" watchObservedRunningTime="2026-11-02 01:57:32.421727029 +0000 UTC m=+1541.634096838"
Nov 02 01:57:32.519466 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:32.519437    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:32.539864 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:32.539812    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8" podStartSLOduration=1.539797611 podStartE2EDuration="1.539797611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 01:57:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 01:57:32.538192523 +0000 UTC m=+1541.750562331" watchObservedRunningTime="2026-11-02 01:57:32.539797611 +0000 UTC m=+1541.752167419"
Nov 02 01:57:32.770902 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:32.770862    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:33.521525 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:33.521478    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xs4xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 02 01:57:33.522018 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:33.521568    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg" podUID="dbce4552-66b5-4a8f-ad14-edacdc87ea52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 02 01:57:33.778673 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:33.778578    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-v8gx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 02 01:57:33.778673 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:33.778635    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8" podUID="18653351-22cb-40db-ae44-1982a38f0a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 02 01:57:34.638733 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:34.590982    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xs4xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 02 01:57:34.638733 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:34.591041    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg" podUID="dbce4552-66b5-4a8f-ad14-edacdc87ea52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 02 01:57:34.782340 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:34.782302    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-v8gx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 02 01:57:34.782549 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:34.782366    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8" podUID="18653351-22cb-40db-ae44-1982a38f0a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 02 01:57:35.519413 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:35.519380    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xs4xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 02 01:57:35.519594 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:35.519430    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg" podUID="dbce4552-66b5-4a8f-ad14-edacdc87ea52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 02 01:57:35.776064 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:35.775967    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-v8gx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 02 01:57:35.776064 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:35.776051    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8" podUID="18653351-22cb-40db-ae44-1982a38f0a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 02 01:57:36.521599 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:36.519592    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xs4xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 02 01:57:36.521599 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:36.519669    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg" podUID="dbce4552-66b5-4a8f-ad14-edacdc87ea52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 02 01:57:36.771140 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:36.771082    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-v8gx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 02 01:57:36.771350 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:36.771228    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8" podUID="18653351-22cb-40db-ae44-1982a38f0a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 02 01:57:37.519828 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:37.519791    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xs4xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 02 01:57:37.520232 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:37.519848    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg" podUID="dbce4552-66b5-4a8f-ad14-edacdc87ea52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 02 01:57:37.778413 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:37.778328    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-v8gx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 02 01:57:37.778672 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:37.778642    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8" podUID="18653351-22cb-40db-ae44-1982a38f0a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 02 01:57:38.520040 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:38.520004    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xs4xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 02 01:57:38.520651 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:38.520618    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg" podUID="dbce4552-66b5-4a8f-ad14-edacdc87ea52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 02 01:57:38.774222 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:38.773479    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-v8gx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 02 01:57:38.774222 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:38.773591    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8" podUID="18653351-22cb-40db-ae44-1982a38f0a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 02 01:57:39.520026 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:39.519988    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xs4xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 02 01:57:39.520196 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:39.520053    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg" podUID="dbce4552-66b5-4a8f-ad14-edacdc87ea52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 02 01:57:39.771577 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:39.771542    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-v8gx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 02 01:57:39.771786 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:39.771591    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8" podUID="18653351-22cb-40db-ae44-1982a38f0a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 02 01:57:39.975029 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:57:39.974985    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:57:41.282455 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:41.282421    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:41.283550 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:41.283469    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:41.284396 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:41.284368    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 01:57:41.498186 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:41.498149    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:41.498611 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:41.498582    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:57:41.499675 ip-10-0-138-75 kubenswrapper[4515]: I1102 01:57:41.499653    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-v8gx8"
Nov 02 01:58:09.959748 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:58:09.959673    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:58:39.959419 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:58:39.959380    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:59:09.959829 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:59:09.959789    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 01:59:39.964166 ip-10-0-138-75 kubenswrapper[4515]: E1102 01:59:39.964106    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:00:00.187043 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.187005    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb"]
Nov 02 02:00:00.191841 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.191821    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n"]
Nov 02 02:00:00.191966 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.191946    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb"
Nov 02 02:00:00.195660 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.195641    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n"
Nov 02 02:00:00.211341 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.211316    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz"]
Nov 02 02:00:00.215396 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.215377    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz"
Nov 02 02:00:00.284174 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.284141    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5mww\" (UniqueName: \"kubernetes.io/projected/93e4de40-ac8b-47b3-9dfb-fd7072257be0-kube-api-access-z5mww\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n\" (UID: \"93e4de40-ac8b-47b3-9dfb-fd7072257be0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n"
Nov 02 02:00:00.284346 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.284301    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bslww\" (UniqueName: \"kubernetes.io/projected/a440be2e-a939-4a5d-a046-93083c782eb0-kube-api-access-bslww\") pod \"maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb\" (UID: \"a440be2e-a939-4a5d-a046-93083c782eb0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb"
Nov 02 02:00:00.284420 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.284345    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e4de40-ac8b-47b3-9dfb-fd7072257be0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n\" (UID: \"93e4de40-ac8b-47b3-9dfb-fd7072257be0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n"
Nov 02 02:00:00.284591 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.284571    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a440be2e-a939-4a5d-a046-93083c782eb0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb\" (UID: \"a440be2e-a939-4a5d-a046-93083c782eb0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb"
Nov 02 02:00:00.318928 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.318894    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n"]
Nov 02 02:00:00.356951 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.356922    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnwgr\""
Nov 02 02:00:00.386665 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.386630    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bslww\" (UniqueName: \"kubernetes.io/projected/a440be2e-a939-4a5d-a046-93083c782eb0-kube-api-access-bslww\") pod \"maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb\" (UID: \"a440be2e-a939-4a5d-a046-93083c782eb0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb"
Nov 02 02:00:00.386895 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.386874    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e4de40-ac8b-47b3-9dfb-fd7072257be0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n\" (UID: \"93e4de40-ac8b-47b3-9dfb-fd7072257be0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n"
Nov 02 02:00:00.387189 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.387161    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c9faeb1-598c-43e1-bbf0-4907010bf722-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz\" (UID: \"5c9faeb1-598c-43e1-bbf0-4907010bf722\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz"
Nov 02 02:00:00.387362 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.387211    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2j97\" (UniqueName: \"kubernetes.io/projected/5c9faeb1-598c-43e1-bbf0-4907010bf722-kube-api-access-r2j97\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz\" (UID: \"5c9faeb1-598c-43e1-bbf0-4907010bf722\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz"
Nov 02 02:00:00.387362 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.387310    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a440be2e-a939-4a5d-a046-93083c782eb0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb\" (UID: \"a440be2e-a939-4a5d-a046-93083c782eb0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb"
Nov 02 02:00:00.387547 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.387385    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5mww\" (UniqueName: \"kubernetes.io/projected/93e4de40-ac8b-47b3-9dfb-fd7072257be0-kube-api-access-z5mww\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n\" (UID: \"93e4de40-ac8b-47b3-9dfb-fd7072257be0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n"
Nov 02 02:00:00.387870 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.387846    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e4de40-ac8b-47b3-9dfb-fd7072257be0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n\" (UID: \"93e4de40-ac8b-47b3-9dfb-fd7072257be0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n"
Nov 02 02:00:00.389470 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.388535    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a440be2e-a939-4a5d-a046-93083c782eb0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb\" (UID: \"a440be2e-a939-4a5d-a046-93083c782eb0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb"
Nov 02 02:00:00.392131 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.392098    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb"]
Nov 02 02:00:00.448262 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.448186    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bslww\" (UniqueName: \"kubernetes.io/projected/a440be2e-a939-4a5d-a046-93083c782eb0-kube-api-access-bslww\") pod \"maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb\" (UID: \"a440be2e-a939-4a5d-a046-93083c782eb0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb"
Nov 02 02:00:00.448839 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.448819    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5mww\" (UniqueName: \"kubernetes.io/projected/93e4de40-ac8b-47b3-9dfb-fd7072257be0-kube-api-access-z5mww\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n\" (UID: \"93e4de40-ac8b-47b3-9dfb-fd7072257be0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n"
Nov 02 02:00:00.488591 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.488562    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c9faeb1-598c-43e1-bbf0-4907010bf722-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz\" (UID: \"5c9faeb1-598c-43e1-bbf0-4907010bf722\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz"
Nov 02 02:00:00.488727 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.488597    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2j97\" (UniqueName: \"kubernetes.io/projected/5c9faeb1-598c-43e1-bbf0-4907010bf722-kube-api-access-r2j97\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz\" (UID: \"5c9faeb1-598c-43e1-bbf0-4907010bf722\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz"
Nov 02 02:00:00.489198 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.489181    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c9faeb1-598c-43e1-bbf0-4907010bf722-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz\" (UID: \"5c9faeb1-598c-43e1-bbf0-4907010bf722\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz"
Nov 02 02:00:00.511019 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.510991    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb"
Nov 02 02:00:00.518946 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.518916    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n"
Nov 02 02:00:00.522403 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.521994    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz"]
Nov 02 02:00:00.551531 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.551504    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2j97\" (UniqueName: \"kubernetes.io/projected/5c9faeb1-598c-43e1-bbf0-4907010bf722-kube-api-access-r2j97\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz\" (UID: \"5c9faeb1-598c-43e1-bbf0-4907010bf722\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz"
Nov 02 02:00:00.827482 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.827392    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz"
Nov 02 02:00:00.914223 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.914187    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n"]
Nov 02 02:00:00.915326 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:00:00.915285    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93e4de40_ac8b_47b3_9dfb_fd7072257be0.slice/crio-7ed5b1e14b10a3e4bdec2abef35c31a1eb43440b5cc9599a71418339863c2a88 WatchSource:0}: Error finding container 7ed5b1e14b10a3e4bdec2abef35c31a1eb43440b5cc9599a71418339863c2a88: Status 404 returned error can't find the container with id 7ed5b1e14b10a3e4bdec2abef35c31a1eb43440b5cc9599a71418339863c2a88
Nov 02 02:00:00.918073 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.918052    4515 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 02:00:00.966604 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:00:00.966573    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda440be2e_a939_4a5d_a046_93083c782eb0.slice/crio-fba0694b0eca78ddccc9f10b5a93a4b3cb7264770656dbbee9e00c3fc0f0970a WatchSource:0}: Error finding container fba0694b0eca78ddccc9f10b5a93a4b3cb7264770656dbbee9e00c3fc0f0970a: Status 404 returned error can't find the container with id fba0694b0eca78ddccc9f10b5a93a4b3cb7264770656dbbee9e00c3fc0f0970a
Nov 02 02:00:00.966931 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:00.966905    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb"]
Nov 02 02:00:01.207898 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:01.207866    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz"]
Nov 02 02:00:01.209028 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:00:01.209005    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c9faeb1_598c_43e1_bbf0_4907010bf722.slice/crio-8f78a7d60ec60bc7052ce1fe4b919e54239f67281770c5ce9559ec88c4b9ef7b WatchSource:0}: Error finding container 8f78a7d60ec60bc7052ce1fe4b919e54239f67281770c5ce9559ec88c4b9ef7b: Status 404 returned error can't find the container with id 8f78a7d60ec60bc7052ce1fe4b919e54239f67281770c5ce9559ec88c4b9ef7b
Nov 02 02:00:01.458061 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:01.457971    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb" event={"ID":"a440be2e-a939-4a5d-a046-93083c782eb0","Type":"ContainerStarted","Data":"fba0694b0eca78ddccc9f10b5a93a4b3cb7264770656dbbee9e00c3fc0f0970a"}
Nov 02 02:00:01.459371 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:01.459345    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz" event={"ID":"5c9faeb1-598c-43e1-bbf0-4907010bf722","Type":"ContainerStarted","Data":"8f78a7d60ec60bc7052ce1fe4b919e54239f67281770c5ce9559ec88c4b9ef7b"}
Nov 02 02:00:01.460668 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:01.460645    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n" event={"ID":"93e4de40-ac8b-47b3-9dfb-fd7072257be0","Type":"ContainerStarted","Data":"7ed5b1e14b10a3e4bdec2abef35c31a1eb43440b5cc9599a71418339863c2a88"}
Nov 02 02:00:09.516798 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:09.516749    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n" event={"ID":"93e4de40-ac8b-47b3-9dfb-fd7072257be0","Type":"ContainerStarted","Data":"60b0697013965c91e9eca2d31ef53e5862898b11502459361588220f6b8ea5b2"}
Nov 02 02:00:09.518437 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:09.518411    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz" event={"ID":"5c9faeb1-598c-43e1-bbf0-4907010bf722","Type":"ContainerStarted","Data":"6a02d18e4590126aa9e058b630ee585f30ac74cc0d47eb29e849bae79f00cdb9"}
Nov 02 02:00:09.634952 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:09.634901    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n" podStartSLOduration=1.692508821 podStartE2EDuration="9.63488494s" totalImagesPullingTime="7.942376119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:00:09.631380752 +0000 UTC m=+1698.843750563" watchObservedRunningTime="2026-11-02 02:00:09.63488494 +0000 UTC m=+1698.847254747"
Nov 02 02:00:09.749639 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:09.749576    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz" podStartSLOduration=2.104590848 podStartE2EDuration="9.74956078s" totalImagesPullingTime="7.644969932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:00:09.746245683 +0000 UTC m=+1698.958615503" watchObservedRunningTime="2026-11-02 02:00:09.74956078 +0000 UTC m=+1698.961930587"
Nov 02 02:00:09.960531 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:00:09.960491    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:00:10.527472 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:10.527439    4515 generic.go:356] "Generic (PLEG): container finished" podID="5c9faeb1-598c-43e1-bbf0-4907010bf722" containerID="6a02d18e4590126aa9e058b630ee585f30ac74cc0d47eb29e849bae79f00cdb9" exitCode=60
Nov 02 02:00:10.527929 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:10.527520    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz" event={"ID":"5c9faeb1-598c-43e1-bbf0-4907010bf722","Type":"ContainerDied","Data":"6a02d18e4590126aa9e058b630ee585f30ac74cc0d47eb29e849bae79f00cdb9"}
Nov 02 02:00:10.527929 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:10.527793    4515 scope.go:122] "RemoveContainer" containerID="6a02d18e4590126aa9e058b630ee585f30ac74cc0d47eb29e849bae79f00cdb9"
Nov 02 02:00:10.532816 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:10.532792    4515 generic.go:356] "Generic (PLEG): container finished" podID="93e4de40-ac8b-47b3-9dfb-fd7072257be0" containerID="60b0697013965c91e9eca2d31ef53e5862898b11502459361588220f6b8ea5b2" exitCode=60
Nov 02 02:00:10.532961 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:10.532937    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n" event={"ID":"93e4de40-ac8b-47b3-9dfb-fd7072257be0","Type":"ContainerDied","Data":"60b0697013965c91e9eca2d31ef53e5862898b11502459361588220f6b8ea5b2"}
Nov 02 02:00:10.533148 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:10.533130    4515 scope.go:122] "RemoveContainer" containerID="60b0697013965c91e9eca2d31ef53e5862898b11502459361588220f6b8ea5b2"
Nov 02 02:00:11.542282 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:11.542250    4515 generic.go:356] "Generic (PLEG): container finished" podID="5c9faeb1-598c-43e1-bbf0-4907010bf722" containerID="0c8f83574ead84de3ab1d43752c045f4b8ea0a1c5a6474d6cc0bccb6a3463a1d" exitCode=60
Nov 02 02:00:11.542643 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:11.542333    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz" event={"ID":"5c9faeb1-598c-43e1-bbf0-4907010bf722","Type":"ContainerDied","Data":"0c8f83574ead84de3ab1d43752c045f4b8ea0a1c5a6474d6cc0bccb6a3463a1d"}
Nov 02 02:00:11.542643 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:11.542407    4515 scope.go:122] "RemoveContainer" containerID="6a02d18e4590126aa9e058b630ee585f30ac74cc0d47eb29e849bae79f00cdb9"
Nov 02 02:00:11.542768 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:11.542750    4515 scope.go:122] "RemoveContainer" containerID="0c8f83574ead84de3ab1d43752c045f4b8ea0a1c5a6474d6cc0bccb6a3463a1d"
Nov 02 02:00:11.543196 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:00:11.543167    4515 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz_odh-ai-gateway-infra(5c9faeb1-598c-43e1-bbf0-4907010bf722)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz" podUID="5c9faeb1-598c-43e1-bbf0-4907010bf722"
Nov 02 02:00:11.544426 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:11.544401    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n" event={"ID":"93e4de40-ac8b-47b3-9dfb-fd7072257be0","Type":"ContainerStarted","Data":"8fc39e32418b9106e0bd813cd0dc293c0a7ed897b61f5d8c6387c00fe01d6841"}
Nov 02 02:00:11.546006 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:11.545985    4515 generic.go:356] "Generic (PLEG): container finished" podID="a440be2e-a939-4a5d-a046-93083c782eb0" containerID="d16a8c84a29532b8a476404be075774d7f2329933a8edfa93f0cfdcd52dc2b92" exitCode=60
Nov 02 02:00:11.546148 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:11.546007    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb" event={"ID":"a440be2e-a939-4a5d-a046-93083c782eb0","Type":"ContainerDied","Data":"d16a8c84a29532b8a476404be075774d7f2329933a8edfa93f0cfdcd52dc2b92"}
Nov 02 02:00:11.546308 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:11.546292    4515 scope.go:122] "RemoveContainer" containerID="d16a8c84a29532b8a476404be075774d7f2329933a8edfa93f0cfdcd52dc2b92"
Nov 02 02:00:12.558479 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:12.558447    4515 scope.go:122] "RemoveContainer" containerID="0c8f83574ead84de3ab1d43752c045f4b8ea0a1c5a6474d6cc0bccb6a3463a1d"
Nov 02 02:00:12.558888 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:00:12.558807    4515 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz_odh-ai-gateway-infra(5c9faeb1-598c-43e1-bbf0-4907010bf722)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz" podUID="5c9faeb1-598c-43e1-bbf0-4907010bf722"
Nov 02 02:00:12.569501 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:12.565486    4515 generic.go:356] "Generic (PLEG): container finished" podID="93e4de40-ac8b-47b3-9dfb-fd7072257be0" containerID="8fc39e32418b9106e0bd813cd0dc293c0a7ed897b61f5d8c6387c00fe01d6841" exitCode=60
Nov 02 02:00:12.569501 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:12.565676    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n" event={"ID":"93e4de40-ac8b-47b3-9dfb-fd7072257be0","Type":"ContainerDied","Data":"8fc39e32418b9106e0bd813cd0dc293c0a7ed897b61f5d8c6387c00fe01d6841"}
Nov 02 02:00:12.569501 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:12.565743    4515 scope.go:122] "RemoveContainer" containerID="60b0697013965c91e9eca2d31ef53e5862898b11502459361588220f6b8ea5b2"
Nov 02 02:00:12.569501 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:12.566214    4515 scope.go:122] "RemoveContainer" containerID="8fc39e32418b9106e0bd813cd0dc293c0a7ed897b61f5d8c6387c00fe01d6841"
Nov 02 02:00:12.569501 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:00:12.566692    4515 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n_odh-ai-gateway-infra(93e4de40-ac8b-47b3-9dfb-fd7072257be0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n" podUID="93e4de40-ac8b-47b3-9dfb-fd7072257be0"
Nov 02 02:00:12.569501 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:12.568958    4515 generic.go:356] "Generic (PLEG): container finished" podID="a440be2e-a939-4a5d-a046-93083c782eb0" containerID="eb455669c33ce9d5c91e27324249e9c78da37b04b439a29f50e602a088c8b8c3" exitCode=60
Nov 02 02:00:12.569501 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:12.569111    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb" event={"ID":"a440be2e-a939-4a5d-a046-93083c782eb0","Type":"ContainerDied","Data":"eb455669c33ce9d5c91e27324249e9c78da37b04b439a29f50e602a088c8b8c3"}
Nov 02 02:00:12.569501 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:12.569421    4515 scope.go:122] "RemoveContainer" containerID="eb455669c33ce9d5c91e27324249e9c78da37b04b439a29f50e602a088c8b8c3"
Nov 02 02:00:12.570162 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:00:12.569759    4515 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb_odh-ai-gateway-infra(a440be2e-a939-4a5d-a046-93083c782eb0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb" podUID="a440be2e-a939-4a5d-a046-93083c782eb0"
Nov 02 02:00:12.592323 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:12.592293    4515 scope.go:122] "RemoveContainer" containerID="d16a8c84a29532b8a476404be075774d7f2329933a8edfa93f0cfdcd52dc2b92"
Nov 02 02:00:13.576802 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:13.576764    4515 scope.go:122] "RemoveContainer" containerID="eb455669c33ce9d5c91e27324249e9c78da37b04b439a29f50e602a088c8b8c3"
Nov 02 02:00:13.577306 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:00:13.577214    4515 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb_odh-ai-gateway-infra(a440be2e-a939-4a5d-a046-93083c782eb0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb" podUID="a440be2e-a939-4a5d-a046-93083c782eb0"
Nov 02 02:00:13.578403 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:13.578381    4515 scope.go:122] "RemoveContainer" containerID="8fc39e32418b9106e0bd813cd0dc293c0a7ed897b61f5d8c6387c00fe01d6841"
Nov 02 02:00:13.578698 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:00:13.578676    4515 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n_odh-ai-gateway-infra(93e4de40-ac8b-47b3-9dfb-fd7072257be0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n" podUID="93e4de40-ac8b-47b3-9dfb-fd7072257be0"
Nov 02 02:00:22.947399 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:22.947370    4515 scope.go:122] "RemoveContainer" containerID="8fc39e32418b9106e0bd813cd0dc293c0a7ed897b61f5d8c6387c00fe01d6841"
Nov 02 02:00:23.640251 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:23.640174    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n" event={"ID":"93e4de40-ac8b-47b3-9dfb-fd7072257be0","Type":"ContainerStarted","Data":"ff4fd10d26215f999a69343c6bdb50915d64d7651f20e1ce10881f25de762ebf"}
Nov 02 02:00:23.947217 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:23.947188    4515 scope.go:122] "RemoveContainer" containerID="0c8f83574ead84de3ab1d43752c045f4b8ea0a1c5a6474d6cc0bccb6a3463a1d"
Nov 02 02:00:24.648270 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:24.648185    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz" event={"ID":"5c9faeb1-598c-43e1-bbf0-4907010bf722","Type":"ContainerStarted","Data":"5971938440c1f1b319048e8d0a7db6cd95dbbf5a53f302b1777e47ed88f8b303"}
Nov 02 02:00:24.650465 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:24.650437    4515 generic.go:356] "Generic (PLEG): container finished" podID="93e4de40-ac8b-47b3-9dfb-fd7072257be0" containerID="ff4fd10d26215f999a69343c6bdb50915d64d7651f20e1ce10881f25de762ebf" exitCode=60
Nov 02 02:00:24.650572 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:24.650506    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n" event={"ID":"93e4de40-ac8b-47b3-9dfb-fd7072257be0","Type":"ContainerDied","Data":"ff4fd10d26215f999a69343c6bdb50915d64d7651f20e1ce10881f25de762ebf"}
Nov 02 02:00:24.650572 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:24.650568    4515 scope.go:122] "RemoveContainer" containerID="8fc39e32418b9106e0bd813cd0dc293c0a7ed897b61f5d8c6387c00fe01d6841"
Nov 02 02:00:24.650952 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:24.650933    4515 scope.go:122] "RemoveContainer" containerID="ff4fd10d26215f999a69343c6bdb50915d64d7651f20e1ce10881f25de762ebf"
Nov 02 02:00:24.651425 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:00:24.651398    4515 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n_odh-ai-gateway-infra(93e4de40-ac8b-47b3-9dfb-fd7072257be0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n" podUID="93e4de40-ac8b-47b3-9dfb-fd7072257be0"
Nov 02 02:00:24.773503 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:24.772661    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n"]
Nov 02 02:00:25.658845 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:25.658808    4515 generic.go:356] "Generic (PLEG): container finished" podID="5c9faeb1-598c-43e1-bbf0-4907010bf722" containerID="5971938440c1f1b319048e8d0a7db6cd95dbbf5a53f302b1777e47ed88f8b303" exitCode=60
Nov 02 02:00:25.659302 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:25.658887    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz" event={"ID":"5c9faeb1-598c-43e1-bbf0-4907010bf722","Type":"ContainerDied","Data":"5971938440c1f1b319048e8d0a7db6cd95dbbf5a53f302b1777e47ed88f8b303"}
Nov 02 02:00:25.659302 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:25.658952    4515 scope.go:122] "RemoveContainer" containerID="0c8f83574ead84de3ab1d43752c045f4b8ea0a1c5a6474d6cc0bccb6a3463a1d"
Nov 02 02:00:25.659302 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:25.659300    4515 scope.go:122] "RemoveContainer" containerID="5971938440c1f1b319048e8d0a7db6cd95dbbf5a53f302b1777e47ed88f8b303"
Nov 02 02:00:25.659617 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:00:25.659592    4515 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz_odh-ai-gateway-infra(5c9faeb1-598c-43e1-bbf0-4907010bf722)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz" podUID="5c9faeb1-598c-43e1-bbf0-4907010bf722"
Nov 02 02:00:25.773542 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:25.773511    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz"]
Nov 02 02:00:25.806888 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:25.806867    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n"
Nov 02 02:00:25.892717 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:25.892689    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93e4de40-ac8b-47b3-9dfb-fd7072257be0-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e4de40-ac8b-47b3-9dfb-fd7072257be0-maas-api-service-ca\") pod \"93e4de40-ac8b-47b3-9dfb-fd7072257be0\" (UID: \"93e4de40-ac8b-47b3-9dfb-fd7072257be0\") "
Nov 02 02:00:25.892915 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:25.892900    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93e4de40-ac8b-47b3-9dfb-fd7072257be0-kube-api-access-z5mww\" (UniqueName: \"kubernetes.io/projected/93e4de40-ac8b-47b3-9dfb-fd7072257be0-kube-api-access-z5mww\") pod \"93e4de40-ac8b-47b3-9dfb-fd7072257be0\" (UID: \"93e4de40-ac8b-47b3-9dfb-fd7072257be0\") "
Nov 02 02:00:25.893094 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:25.893072    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93e4de40-ac8b-47b3-9dfb-fd7072257be0-maas-api-service-ca" pod "93e4de40-ac8b-47b3-9dfb-fd7072257be0" (UID: "93e4de40-ac8b-47b3-9dfb-fd7072257be0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:00:25.893413 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:25.893393    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e4de40-ac8b-47b3-9dfb-fd7072257be0-maas-api-service-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:25.894898 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:25.894879    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93e4de40-ac8b-47b3-9dfb-fd7072257be0-kube-api-access-z5mww" pod "93e4de40-ac8b-47b3-9dfb-fd7072257be0" (UID: "93e4de40-ac8b-47b3-9dfb-fd7072257be0"). InnerVolumeSpecName "kube-api-access-z5mww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:00:25.946773 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:25.946750    4515 scope.go:122] "RemoveContainer" containerID="eb455669c33ce9d5c91e27324249e9c78da37b04b439a29f50e602a088c8b8c3"
Nov 02 02:00:25.995096 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:25.995072    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5mww\" (UniqueName: \"kubernetes.io/projected/93e4de40-ac8b-47b3-9dfb-fd7072257be0-kube-api-access-z5mww\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:26.729130 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:26.727821    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb" event={"ID":"a440be2e-a939-4a5d-a046-93083c782eb0","Type":"ContainerStarted","Data":"f127a167ee60215f9d9b6c54638af8b0b0e311d5ad44d69306dd17439c0db22a"}
Nov 02 02:00:26.738553 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:26.738369    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n" event={"ID":"93e4de40-ac8b-47b3-9dfb-fd7072257be0","Type":"ContainerDied","Data":"7ed5b1e14b10a3e4bdec2abef35c31a1eb43440b5cc9599a71418339863c2a88"}
Nov 02 02:00:26.738858 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:26.738835    4515 scope.go:122] "RemoveContainer" containerID="ff4fd10d26215f999a69343c6bdb50915d64d7651f20e1ce10881f25de762ebf"
Nov 02 02:00:26.739068 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:26.739051    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n"
Nov 02 02:00:27.013533 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.009347    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb" podStartSLOduration=17.585296839 podStartE2EDuration="27.009324784s" totalImagesPullingTime="9.424027945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:00:26.871451498 +0000 UTC m=+1716.083821324" watchObservedRunningTime="2026-11-02 02:00:27.009324784 +0000 UTC m=+1716.221694593"
Nov 02 02:00:27.023029 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.022377    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n"]
Nov 02 02:00:27.025946 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.025515    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-606ca0-29893080-w6r7n"]
Nov 02 02:00:27.041138 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.040817    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz"
Nov 02 02:00:27.128868 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.128832    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c9faeb1-598c-43e1-bbf0-4907010bf722-kube-api-access-r2j97\" (UniqueName: \"kubernetes.io/projected/5c9faeb1-598c-43e1-bbf0-4907010bf722-kube-api-access-r2j97\") pod \"5c9faeb1-598c-43e1-bbf0-4907010bf722\" (UID: \"5c9faeb1-598c-43e1-bbf0-4907010bf722\") "
Nov 02 02:00:27.129189 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.129172    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c9faeb1-598c-43e1-bbf0-4907010bf722-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c9faeb1-598c-43e1-bbf0-4907010bf722-maas-api-service-ca\") pod \"5c9faeb1-598c-43e1-bbf0-4907010bf722\" (UID: \"5c9faeb1-598c-43e1-bbf0-4907010bf722\") "
Nov 02 02:00:27.129559 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.129528    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c9faeb1-598c-43e1-bbf0-4907010bf722-maas-api-service-ca" pod "5c9faeb1-598c-43e1-bbf0-4907010bf722" (UID: "5c9faeb1-598c-43e1-bbf0-4907010bf722"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:00:27.129777 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.129759    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c9faeb1-598c-43e1-bbf0-4907010bf722-maas-api-service-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:27.131230 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.131205    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c9faeb1-598c-43e1-bbf0-4907010bf722-kube-api-access-r2j97" pod "5c9faeb1-598c-43e1-bbf0-4907010bf722" (UID: "5c9faeb1-598c-43e1-bbf0-4907010bf722"). InnerVolumeSpecName "kube-api-access-r2j97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:00:27.230898 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.230860    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2j97\" (UniqueName: \"kubernetes.io/projected/5c9faeb1-598c-43e1-bbf0-4907010bf722-kube-api-access-r2j97\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:27.746737 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.746692    4515 generic.go:356] "Generic (PLEG): container finished" podID="a440be2e-a939-4a5d-a046-93083c782eb0" containerID="f127a167ee60215f9d9b6c54638af8b0b0e311d5ad44d69306dd17439c0db22a" exitCode=60
Nov 02 02:00:27.747229 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.746751    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb" event={"ID":"a440be2e-a939-4a5d-a046-93083c782eb0","Type":"ContainerDied","Data":"f127a167ee60215f9d9b6c54638af8b0b0e311d5ad44d69306dd17439c0db22a"}
Nov 02 02:00:27.747229 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.746858    4515 scope.go:122] "RemoveContainer" containerID="eb455669c33ce9d5c91e27324249e9c78da37b04b439a29f50e602a088c8b8c3"
Nov 02 02:00:27.747356 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.747280    4515 scope.go:122] "RemoveContainer" containerID="f127a167ee60215f9d9b6c54638af8b0b0e311d5ad44d69306dd17439c0db22a"
Nov 02 02:00:27.747648 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:00:27.747620    4515 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb_odh-ai-gateway-infra(a440be2e-a939-4a5d-a046-93083c782eb0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb" podUID="a440be2e-a939-4a5d-a046-93083c782eb0"
Nov 02 02:00:27.748596 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.748575    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz" event={"ID":"5c9faeb1-598c-43e1-bbf0-4907010bf722","Type":"ContainerDied","Data":"8f78a7d60ec60bc7052ce1fe4b919e54239f67281770c5ce9559ec88c4b9ef7b"}
Nov 02 02:00:27.748692 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.748599    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz"
Nov 02 02:00:27.760097 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.760065    4515 scope.go:122] "RemoveContainer" containerID="5971938440c1f1b319048e8d0a7db6cd95dbbf5a53f302b1777e47ed88f8b303"
Nov 02 02:00:27.855410 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.855371    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb"]
Nov 02 02:00:27.998590 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:27.998517    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz"]
Nov 02 02:00:28.004812 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:28.004781    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-73e1cc-29893080-k89pz"]
Nov 02 02:00:28.933867 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:28.933846    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb"
Nov 02 02:00:28.954093 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:28.954065    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c9faeb1-598c-43e1-bbf0-4907010bf722" path="/var/lib/kubelet/pods/5c9faeb1-598c-43e1-bbf0-4907010bf722/volumes"
Nov 02 02:00:28.954578 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:28.954562    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93e4de40-ac8b-47b3-9dfb-fd7072257be0" path="/var/lib/kubelet/pods/93e4de40-ac8b-47b3-9dfb-fd7072257be0/volumes"
Nov 02 02:00:29.058819 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:29.057828    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a440be2e-a939-4a5d-a046-93083c782eb0-kube-api-access-bslww\" (UniqueName: \"kubernetes.io/projected/a440be2e-a939-4a5d-a046-93083c782eb0-kube-api-access-bslww\") pod \"a440be2e-a939-4a5d-a046-93083c782eb0\" (UID: \"a440be2e-a939-4a5d-a046-93083c782eb0\") "
Nov 02 02:00:29.058819 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:29.058102    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a440be2e-a939-4a5d-a046-93083c782eb0-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a440be2e-a939-4a5d-a046-93083c782eb0-maas-api-service-ca\") pod \"a440be2e-a939-4a5d-a046-93083c782eb0\" (UID: \"a440be2e-a939-4a5d-a046-93083c782eb0\") "
Nov 02 02:00:29.058819 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:29.058563    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a440be2e-a939-4a5d-a046-93083c782eb0-maas-api-service-ca" pod "a440be2e-a939-4a5d-a046-93083c782eb0" (UID: "a440be2e-a939-4a5d-a046-93083c782eb0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:00:29.058819 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:29.058770    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a440be2e-a939-4a5d-a046-93083c782eb0-maas-api-service-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:29.061257 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:29.061220    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a440be2e-a939-4a5d-a046-93083c782eb0-kube-api-access-bslww" pod "a440be2e-a939-4a5d-a046-93083c782eb0" (UID: "a440be2e-a939-4a5d-a046-93083c782eb0"). InnerVolumeSpecName "kube-api-access-bslww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:00:29.159964 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:29.159929    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bslww\" (UniqueName: \"kubernetes.io/projected/a440be2e-a939-4a5d-a046-93083c782eb0-kube-api-access-bslww\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:29.770241 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:29.770210    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb" event={"ID":"a440be2e-a939-4a5d-a046-93083c782eb0","Type":"ContainerDied","Data":"fba0694b0eca78ddccc9f10b5a93a4b3cb7264770656dbbee9e00c3fc0f0970a"}
Nov 02 02:00:29.770432 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:29.770268    4515 scope.go:122] "RemoveContainer" containerID="f127a167ee60215f9d9b6c54638af8b0b0e311d5ad44d69306dd17439c0db22a"
Nov 02 02:00:29.770432 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:29.770224    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb"
Nov 02 02:00:29.934513 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:29.934478    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb"]
Nov 02 02:00:29.938618 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:29.938590    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c62671-29893080-fb9pb"]
Nov 02 02:00:30.954047 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:30.954017    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a440be2e-a939-4a5d-a046-93083c782eb0" path="/var/lib/kubelet/pods/a440be2e-a939-4a5d-a046-93083c782eb0/volumes"
Nov 02 02:00:39.960165 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:00:39.960110    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:00:41.453622 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:41.453581    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"]
Nov 02 02:00:41.454414 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:41.454144    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq" podUID="90e69d4e-ddd4-4ec5-b334-40f7a882731b" containerName="istio-proxy" containerID="cri-o://f7e91945dd61064976deb30e7076fa929dbe1322555b13ae73f029267fe720bc" gracePeriod=30
Nov 02 02:00:41.478507 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:41.478464    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"]
Nov 02 02:00:41.478807 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:41.478773    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv" podUID="07f6d265-73ef-481c-b583-73c648e7747c" containerName="istio-proxy" containerID="cri-o://160d48e249e68bcffe3e820edd7ca58c4ef17a6ecb94d0b014e8168b78e74a15" gracePeriod=30
Nov 02 02:00:46.692173 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.692136    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 02 02:00:46.692625 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.692210    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv" podUID="07f6d265-73ef-481c-b583-73c648e7747c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 02 02:00:46.766922 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.766899    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 02:00:46.791878 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.791855    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.867590    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07f6d265-73ef-481c-b583-73c648e7747c-istio-token\" (UniqueName: \"kubernetes.io/projected/07f6d265-73ef-481c-b583-73c648e7747c-istio-token\") pod \"07f6d265-73ef-481c-b583-73c648e7747c\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") "
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.867647    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-workload-socket\") pod \"07f6d265-73ef-481c-b583-73c648e7747c\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") "
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.867703    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07f6d265-73ef-481c-b583-73c648e7747c-kube-api-access-278fl\" (UniqueName: \"kubernetes.io/projected/07f6d265-73ef-481c-b583-73c648e7747c-kube-api-access-278fl\") pod \"07f6d265-73ef-481c-b583-73c648e7747c\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") "
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.867735    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istiod-ca-cert\") pod \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") "
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.867762    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-credential-socket\") pod \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") "
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.867794    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90e69d4e-ddd4-4ec5-b334-40f7a882731b-kube-api-access-5k64s\" (UniqueName: \"kubernetes.io/projected/90e69d4e-ddd4-4ec5-b334-40f7a882731b-kube-api-access-5k64s\") pod \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") "
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.867936    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-istio-data\") pod \"07f6d265-73ef-481c-b583-73c648e7747c\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") "
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.867989    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-workload-certs\") pod \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") "
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.868019    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-istio-envoy\") pod \"07f6d265-73ef-481c-b583-73c648e7747c\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") "
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.868048    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-envoy\") pod \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") "
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.868132    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-data\") pod \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") "
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.868170    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-credential-socket\") pod \"07f6d265-73ef-481c-b583-73c648e7747c\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") "
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.868330    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/07f6d265-73ef-481c-b583-73c648e7747c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f6d265-73ef-481c-b583-73c648e7747c-istiod-ca-cert\") pod \"07f6d265-73ef-481c-b583-73c648e7747c\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") "
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.868376    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-token\" (UniqueName: \"kubernetes.io/projected/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-token\") pod \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") "
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.868411    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-workload-socket\") pod \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") "
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.868448    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/07f6d265-73ef-481c-b583-73c648e7747c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f6d265-73ef-481c-b583-73c648e7747c-istio-podinfo\") pod \"07f6d265-73ef-481c-b583-73c648e7747c\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") "
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.868478    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-podinfo\") pod \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\" (UID: \"90e69d4e-ddd4-4ec5-b334-40f7a882731b\") "
Nov 02 02:00:46.874155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.868543    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-workload-certs\") pod \"07f6d265-73ef-481c-b583-73c648e7747c\" (UID: \"07f6d265-73ef-481c-b583-73c648e7747c\") "
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.879459    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f6d265-73ef-481c-b583-73c648e7747c-istio-token" pod "07f6d265-73ef-481c-b583-73c648e7747c" (UID: "07f6d265-73ef-481c-b583-73c648e7747c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.879722    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-workload-socket" pod "07f6d265-73ef-481c-b583-73c648e7747c" (UID: "07f6d265-73ef-481c-b583-73c648e7747c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.882755    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f6d265-73ef-481c-b583-73c648e7747c-kube-api-access-278fl" pod "07f6d265-73ef-481c-b583-73c648e7747c" (UID: "07f6d265-73ef-481c-b583-73c648e7747c"). InnerVolumeSpecName "kube-api-access-278fl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.883228    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istiod-ca-cert" pod "90e69d4e-ddd4-4ec5-b334-40f7a882731b" (UID: "90e69d4e-ddd4-4ec5-b334-40f7a882731b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.883562    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-credential-socket" pod "90e69d4e-ddd4-4ec5-b334-40f7a882731b" (UID: "90e69d4e-ddd4-4ec5-b334-40f7a882731b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.885995    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90e69d4e-ddd4-4ec5-b334-40f7a882731b-kube-api-access-5k64s" pod "90e69d4e-ddd4-4ec5-b334-40f7a882731b" (UID: "90e69d4e-ddd4-4ec5-b334-40f7a882731b"). InnerVolumeSpecName "kube-api-access-5k64s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.886262    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-istio-data" pod "07f6d265-73ef-481c-b583-73c648e7747c" (UID: "07f6d265-73ef-481c-b583-73c648e7747c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.886473    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-workload-certs" pod "90e69d4e-ddd4-4ec5-b334-40f7a882731b" (UID: "90e69d4e-ddd4-4ec5-b334-40f7a882731b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.888844    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-istio-envoy" pod "07f6d265-73ef-481c-b583-73c648e7747c" (UID: "07f6d265-73ef-481c-b583-73c648e7747c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.891038    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-envoy" pod "90e69d4e-ddd4-4ec5-b334-40f7a882731b" (UID: "90e69d4e-ddd4-4ec5-b334-40f7a882731b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.891346    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-data" pod "90e69d4e-ddd4-4ec5-b334-40f7a882731b" (UID: "90e69d4e-ddd4-4ec5-b334-40f7a882731b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.891527    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-workload-socket" pod "90e69d4e-ddd4-4ec5-b334-40f7a882731b" (UID: "90e69d4e-ddd4-4ec5-b334-40f7a882731b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.891731    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-credential-socket" pod "07f6d265-73ef-481c-b583-73c648e7747c" (UID: "07f6d265-73ef-481c-b583-73c648e7747c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.892467    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07f6d265-73ef-481c-b583-73c648e7747c-istiod-ca-cert" pod "07f6d265-73ef-481c-b583-73c648e7747c" (UID: "07f6d265-73ef-481c-b583-73c648e7747c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.893380    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-workload-certs" pod "07f6d265-73ef-481c-b583-73c648e7747c" (UID: "07f6d265-73ef-481c-b583-73c648e7747c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.898455    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07f6d265-73ef-481c-b583-73c648e7747c-istio-podinfo" pod "07f6d265-73ef-481c-b583-73c648e7747c" (UID: "07f6d265-73ef-481c-b583-73c648e7747c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.899697    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-token" pod "90e69d4e-ddd4-4ec5-b334-40f7a882731b" (UID: "90e69d4e-ddd4-4ec5-b334-40f7a882731b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:00:46.906157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.901271    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-podinfo" pod "90e69d4e-ddd4-4ec5-b334-40f7a882731b" (UID: "90e69d4e-ddd4-4ec5-b334-40f7a882731b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:00:46.958457 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.958410    4515 generic.go:356] "Generic (PLEG): container finished" podID="90e69d4e-ddd4-4ec5-b334-40f7a882731b" containerID="f7e91945dd61064976deb30e7076fa929dbe1322555b13ae73f029267fe720bc" exitCode=0
Nov 02 02:00:46.958659 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.958641    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"
Nov 02 02:00:46.967186 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.966028    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq" event={"ID":"90e69d4e-ddd4-4ec5-b334-40f7a882731b","Type":"ContainerDied","Data":"f7e91945dd61064976deb30e7076fa929dbe1322555b13ae73f029267fe720bc"}
Nov 02 02:00:46.967186 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.966084    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq" event={"ID":"90e69d4e-ddd4-4ec5-b334-40f7a882731b","Type":"ContainerDied","Data":"303fb418eab6a33a2b75eec99ee6c16aba9f92b0c14fba244a2573d576c09b4b"}
Nov 02 02:00:46.967186 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.966130    4515 scope.go:122] "RemoveContainer" containerID="f7e91945dd61064976deb30e7076fa929dbe1322555b13ae73f029267fe720bc"
Nov 02 02:00:46.970471 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970444    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.970602 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970474    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.970602 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970490    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.970602 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970504    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.970602 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970518    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.970602 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970532    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.970602 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970546    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f6d265-73ef-481c-b583-73c648e7747c-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.970602 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970558    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.970602 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970571    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.970602 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970585    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f6d265-73ef-481c-b583-73c648e7747c-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.970602 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970598    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.970973 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970611    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.970973 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970625    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f6d265-73ef-481c-b583-73c648e7747c-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.970973 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970638    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6d265-73ef-481c-b583-73c648e7747c-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.970973 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970652    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-278fl\" (UniqueName: \"kubernetes.io/projected/07f6d265-73ef-481c-b583-73c648e7747c-kube-api-access-278fl\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.970973 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970665    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90e69d4e-ddd4-4ec5-b334-40f7a882731b-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.970973 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970678    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e69d4e-ddd4-4ec5-b334-40f7a882731b-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.970973 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.970694    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5k64s\" (UniqueName: \"kubernetes.io/projected/90e69d4e-ddd4-4ec5-b334-40f7a882731b-kube-api-access-5k64s\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:00:46.972496 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.972441    4515 generic.go:356] "Generic (PLEG): container finished" podID="07f6d265-73ef-481c-b583-73c648e7747c" containerID="160d48e249e68bcffe3e820edd7ca58c4ef17a6ecb94d0b014e8168b78e74a15" exitCode=0
Nov 02 02:00:46.972612 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.972505    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv" event={"ID":"07f6d265-73ef-481c-b583-73c648e7747c","Type":"ContainerDied","Data":"160d48e249e68bcffe3e820edd7ca58c4ef17a6ecb94d0b014e8168b78e74a15"}
Nov 02 02:00:46.972612 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.972524    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"
Nov 02 02:00:46.972612 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.972576    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv" event={"ID":"07f6d265-73ef-481c-b583-73c648e7747c","Type":"ContainerDied","Data":"b82767556ea1a8f1eb1f5e6fed555bec13463e9106355ce30433166f68711418"}
Nov 02 02:00:46.985072 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.985051    4515 scope.go:122] "RemoveContainer" containerID="f7e91945dd61064976deb30e7076fa929dbe1322555b13ae73f029267fe720bc"
Nov 02 02:00:46.985445 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:00:46.985418    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7e91945dd61064976deb30e7076fa929dbe1322555b13ae73f029267fe720bc\": container with ID starting with f7e91945dd61064976deb30e7076fa929dbe1322555b13ae73f029267fe720bc not found: ID does not exist" containerID="f7e91945dd61064976deb30e7076fa929dbe1322555b13ae73f029267fe720bc"
Nov 02 02:00:46.985530 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.985452    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7e91945dd61064976deb30e7076fa929dbe1322555b13ae73f029267fe720bc"} err="failed to get container status \"f7e91945dd61064976deb30e7076fa929dbe1322555b13ae73f029267fe720bc\": rpc error: code = NotFound desc = could not find container \"f7e91945dd61064976deb30e7076fa929dbe1322555b13ae73f029267fe720bc\": container with ID starting with f7e91945dd61064976deb30e7076fa929dbe1322555b13ae73f029267fe720bc not found: ID does not exist"
Nov 02 02:00:46.985530 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.985469    4515 scope.go:122] "RemoveContainer" containerID="160d48e249e68bcffe3e820edd7ca58c4ef17a6ecb94d0b014e8168b78e74a15"
Nov 02 02:00:46.994218 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.994193    4515 scope.go:122] "RemoveContainer" containerID="160d48e249e68bcffe3e820edd7ca58c4ef17a6ecb94d0b014e8168b78e74a15"
Nov 02 02:00:46.994460 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:00:46.994438    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"160d48e249e68bcffe3e820edd7ca58c4ef17a6ecb94d0b014e8168b78e74a15\": container with ID starting with 160d48e249e68bcffe3e820edd7ca58c4ef17a6ecb94d0b014e8168b78e74a15 not found: ID does not exist" containerID="160d48e249e68bcffe3e820edd7ca58c4ef17a6ecb94d0b014e8168b78e74a15"
Nov 02 02:00:46.994530 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:46.994469    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"160d48e249e68bcffe3e820edd7ca58c4ef17a6ecb94d0b014e8168b78e74a15"} err="failed to get container status \"160d48e249e68bcffe3e820edd7ca58c4ef17a6ecb94d0b014e8168b78e74a15\": rpc error: code = NotFound desc = could not find container \"160d48e249e68bcffe3e820edd7ca58c4ef17a6ecb94d0b014e8168b78e74a15\": container with ID starting with 160d48e249e68bcffe3e820edd7ca58c4ef17a6ecb94d0b014e8168b78e74a15 not found: ID does not exist"
Nov 02 02:00:47.191448 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:47.191408    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"]
Nov 02 02:00:47.194623 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:47.194597    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-49zwq"]
Nov 02 02:00:47.300642 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:47.300536    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"]
Nov 02 02:00:47.305039 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:47.305012    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-n7mwv"]
Nov 02 02:00:48.964162 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:48.964092    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07f6d265-73ef-481c-b583-73c648e7747c" path="/var/lib/kubelet/pods/07f6d265-73ef-481c-b583-73c648e7747c/volumes"
Nov 02 02:00:48.964881 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:00:48.964854    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90e69d4e-ddd4-4ec5-b334-40f7a882731b" path="/var/lib/kubelet/pods/90e69d4e-ddd4-4ec5-b334-40f7a882731b/volumes"
Nov 02 02:01:09.958100 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:01:09.958063    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:01:39.958956 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:01:39.958921    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:01:51.147336 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:01:51.147308    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 02:01:51.150739 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:01:51.150719    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 02:02:09.961473 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:02:09.961437    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:02:40.073353 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:02:40.073314    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:02:45.363263 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363225    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"]
Nov 02 02:02:45.363738 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363724    4515 cpu_manager.go:414] "containerMap: removing container" podUID="90e69d4e-ddd4-4ec5-b334-40f7a882731b" containerName="istio-proxy"
Nov 02 02:02:45.363782 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363739    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90e69d4e-ddd4-4ec5-b334-40f7a882731b" containerName="istio-proxy"
Nov 02 02:02:45.363782 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363748    4515 cpu_manager.go:414] "containerMap: removing container" podUID="5c9faeb1-598c-43e1-bbf0-4907010bf722" containerName="cleanup"
Nov 02 02:02:45.363782 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363755    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c9faeb1-598c-43e1-bbf0-4907010bf722" containerName="cleanup"
Nov 02 02:02:45.363782 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363765    4515 cpu_manager.go:414] "containerMap: removing container" podUID="a440be2e-a939-4a5d-a046-93083c782eb0" containerName="cleanup"
Nov 02 02:02:45.363782 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363771    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a440be2e-a939-4a5d-a046-93083c782eb0" containerName="cleanup"
Nov 02 02:02:45.363782 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363780    4515 cpu_manager.go:414] "containerMap: removing container" podUID="93e4de40-ac8b-47b3-9dfb-fd7072257be0" containerName="cleanup"
Nov 02 02:02:45.363782 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363785    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93e4de40-ac8b-47b3-9dfb-fd7072257be0" containerName="cleanup"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363795    4515 cpu_manager.go:414] "containerMap: removing container" podUID="5c9faeb1-598c-43e1-bbf0-4907010bf722" containerName="cleanup"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363799    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c9faeb1-598c-43e1-bbf0-4907010bf722" containerName="cleanup"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363816    4515 cpu_manager.go:414] "containerMap: removing container" podUID="5c9faeb1-598c-43e1-bbf0-4907010bf722" containerName="cleanup"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363821    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c9faeb1-598c-43e1-bbf0-4907010bf722" containerName="cleanup"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363829    4515 cpu_manager.go:414] "containerMap: removing container" podUID="07f6d265-73ef-481c-b583-73c648e7747c" containerName="istio-proxy"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363835    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07f6d265-73ef-481c-b583-73c648e7747c" containerName="istio-proxy"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363842    4515 cpu_manager.go:414] "containerMap: removing container" podUID="a440be2e-a939-4a5d-a046-93083c782eb0" containerName="cleanup"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363847    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a440be2e-a939-4a5d-a046-93083c782eb0" containerName="cleanup"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363853    4515 cpu_manager.go:414] "containerMap: removing container" podUID="93e4de40-ac8b-47b3-9dfb-fd7072257be0" containerName="cleanup"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363857    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93e4de40-ac8b-47b3-9dfb-fd7072257be0" containerName="cleanup"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363865    4515 cpu_manager.go:414] "containerMap: removing container" podUID="93e4de40-ac8b-47b3-9dfb-fd7072257be0" containerName="cleanup"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363869    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93e4de40-ac8b-47b3-9dfb-fd7072257be0" containerName="cleanup"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363941    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c9faeb1-598c-43e1-bbf0-4907010bf722" containerName="cleanup"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363949    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="a440be2e-a939-4a5d-a046-93083c782eb0" containerName="cleanup"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363956    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c9faeb1-598c-43e1-bbf0-4907010bf722" containerName="cleanup"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363962    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="07f6d265-73ef-481c-b583-73c648e7747c" containerName="istio-proxy"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363967    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="a440be2e-a939-4a5d-a046-93083c782eb0" containerName="cleanup"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363972    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="93e4de40-ac8b-47b3-9dfb-fd7072257be0" containerName="cleanup"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363977    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="93e4de40-ac8b-47b3-9dfb-fd7072257be0" containerName="cleanup"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363984    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="90e69d4e-ddd4-4ec5-b334-40f7a882731b" containerName="istio-proxy"
Nov 02 02:02:45.364020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.363989    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="93e4de40-ac8b-47b3-9dfb-fd7072257be0" containerName="cleanup"
Nov 02 02:02:45.367384 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.367358    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.419743 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.419712    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-c1aad5-openshift-default-dockercfg-8lcpp\""
Nov 02 02:02:45.472146 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.469269    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79ffb456-7b20-4afc-8442-0772cf97b269-istiod-ca-cert\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.472146 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.469331    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79ffb456-7b20-4afc-8442-0772cf97b269-istio-podinfo\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.472146 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.469368    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-istio-envoy\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.472146 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.469466    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvwt8\" (UniqueName: \"kubernetes.io/projected/79ffb456-7b20-4afc-8442-0772cf97b269-kube-api-access-vvwt8\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.472146 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.469528    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-workload-certs\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.472146 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.469688    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-istio-data\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.472146 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.469757    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-credential-socket\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.472146 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.469829    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-workload-socket\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.472146 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.469875    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79ffb456-7b20-4afc-8442-0772cf97b269-istio-token\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.474449 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.474409    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"]
Nov 02 02:02:45.570912 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.570866    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-istio-data\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.571102 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.570952    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-credential-socket\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.571102 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.571007    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-workload-socket\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.571102 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.571047    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79ffb456-7b20-4afc-8442-0772cf97b269-istio-token\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.571102 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.571077    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79ffb456-7b20-4afc-8442-0772cf97b269-istiod-ca-cert\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.571394 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.571104    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79ffb456-7b20-4afc-8442-0772cf97b269-istio-podinfo\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.571394 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.571152    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-istio-envoy\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.571394 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.571236    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvwt8\" (UniqueName: \"kubernetes.io/projected/79ffb456-7b20-4afc-8442-0772cf97b269-kube-api-access-vvwt8\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.571394 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.571293    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-workload-certs\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.571394 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.571350    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-credential-socket\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.571394 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.571350    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-istio-data\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.571890 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.571861    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79ffb456-7b20-4afc-8442-0772cf97b269-istiod-ca-cert\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.572013 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.571987    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-workload-socket\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.572194 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.572133    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-workload-certs\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.573764 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.573744    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-istio-envoy\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.573854 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.573836    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79ffb456-7b20-4afc-8442-0772cf97b269-istio-podinfo\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.632370 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.632294    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79ffb456-7b20-4afc-8442-0772cf97b269-istio-token\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.632667 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.632641    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvwt8\" (UniqueName: \"kubernetes.io/projected/79ffb456-7b20-4afc-8442-0772cf97b269-kube-api-access-vvwt8\") pod \"e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") " pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:45.684776 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:45.684736    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:46.124612 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:46.124584    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"]
Nov 02 02:02:46.126771 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:02:46.126735    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ffb456_7b20_4afc_8442_0772cf97b269.slice/crio-b2a91d76178e0fcc5924155c7692efb8779d7de9018ce50fba0fe265aad354cd WatchSource:0}: Error finding container b2a91d76178e0fcc5924155c7692efb8779d7de9018ce50fba0fe265aad354cd: Status 404 returned error can't find the container with id b2a91d76178e0fcc5924155c7692efb8779d7de9018ce50fba0fe265aad354cd
Nov 02 02:02:46.129198 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:46.129164    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:02:46.129291 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:46.129239    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:02:46.129291 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:46.129269    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:02:46.812652 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:46.812613    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2" event={"ID":"79ffb456-7b20-4afc-8442-0772cf97b269","Type":"ContainerStarted","Data":"8f84956af8d7b42b4a9fb9cd9fe3276777ea7d588b4d562eb4885c794467dd7d"}
Nov 02 02:02:46.813112 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:46.812674    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2" event={"ID":"79ffb456-7b20-4afc-8442-0772cf97b269","Type":"ContainerStarted","Data":"b2a91d76178e0fcc5924155c7692efb8779d7de9018ce50fba0fe265aad354cd"}
Nov 02 02:02:46.933701 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:46.933649    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2" podStartSLOduration=1.933634902 podStartE2EDuration="1.933634902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:02:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:02:46.931631109 +0000 UTC m=+1856.144000980" watchObservedRunningTime="2026-11-02 02:02:46.933634902 +0000 UTC m=+1856.146004785"
Nov 02 02:02:47.685896 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:47.685865    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:47.687098 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:47.687072    4515 patch_prober.go:29] interesting pod/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 02:02:47.687229 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:47.687144    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2" podUID="79ffb456-7b20-4afc-8442-0772cf97b269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 02:02:48.685525 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:48.685495    4515 patch_prober.go:29] interesting pod/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 02:02:48.685913 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:48.685559    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2" podUID="79ffb456-7b20-4afc-8442-0772cf97b269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 02:02:49.686432 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:49.686305    4515 patch_prober.go:29] interesting pod/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 02:02:49.686432 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:49.686364    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2" podUID="79ffb456-7b20-4afc-8442-0772cf97b269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 02:02:50.716713 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:50.715555    4515 patch_prober.go:29] interesting pod/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 02:02:50.716713 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:50.715635    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2" podUID="79ffb456-7b20-4afc-8442-0772cf97b269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 02:02:51.685293 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:51.685255    4515 patch_prober.go:29] interesting pod/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 02:02:51.685461 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:51.685324    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2" podUID="79ffb456-7b20-4afc-8442-0772cf97b269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 02:02:52.686391 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:52.686351    4515 patch_prober.go:29] interesting pod/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 02:02:52.687781 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:52.687743    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2" podUID="79ffb456-7b20-4afc-8442-0772cf97b269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 02:02:53.690355 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:53.690325    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:53.690840 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:53.690678    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:02:53.691458 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:02:53.691441    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:03:05.766836 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:05.764722    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"]
Nov 02 02:03:05.766836 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:05.764992    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk" podUID="df6ef068-9dda-4168-b5ba-1768306ddb74" containerName="istio-proxy" containerID="cri-o://2720db1774a79797afc107bf72b6a9127aeecd8a60a82b54858f74875e668734" gracePeriod=30
Nov 02 02:03:09.958987 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:03:09.958946    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:03:11.043937 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.043897    4515 generic.go:356] "Generic (PLEG): container finished" podID="df6ef068-9dda-4168-b5ba-1768306ddb74" containerID="2720db1774a79797afc107bf72b6a9127aeecd8a60a82b54858f74875e668734" exitCode=0
Nov 02 02:03:11.044476 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.043953    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk" event={"ID":"df6ef068-9dda-4168-b5ba-1768306ddb74","Type":"ContainerDied","Data":"2720db1774a79797afc107bf72b6a9127aeecd8a60a82b54858f74875e668734"}
Nov 02 02:03:11.110161 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.110137    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 02:03:11.202474 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.202443    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-workload-certs\") pod \"df6ef068-9dda-4168-b5ba-1768306ddb74\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") "
Nov 02 02:03:11.202614 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.202524    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df6ef068-9dda-4168-b5ba-1768306ddb74-kube-api-access-b2qps\" (UniqueName: \"kubernetes.io/projected/df6ef068-9dda-4168-b5ba-1768306ddb74-kube-api-access-b2qps\") pod \"df6ef068-9dda-4168-b5ba-1768306ddb74\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") "
Nov 02 02:03:11.202614 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.202593    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-data\") pod \"df6ef068-9dda-4168-b5ba-1768306ddb74\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") "
Nov 02 02:03:11.202720 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.202622    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df6ef068-9dda-4168-b5ba-1768306ddb74-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df6ef068-9dda-4168-b5ba-1768306ddb74-istiod-ca-cert\") pod \"df6ef068-9dda-4168-b5ba-1768306ddb74\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") "
Nov 02 02:03:11.202720 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.202656    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-credential-socket\") pod \"df6ef068-9dda-4168-b5ba-1768306ddb74\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") "
Nov 02 02:03:11.202720 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.202684    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-podinfo\") pod \"df6ef068-9dda-4168-b5ba-1768306ddb74\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") "
Nov 02 02:03:11.202720 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.202712    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-envoy\") pod \"df6ef068-9dda-4168-b5ba-1768306ddb74\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") "
Nov 02 02:03:11.202929 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.202723    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-workload-certs" pod "df6ef068-9dda-4168-b5ba-1768306ddb74" (UID: "df6ef068-9dda-4168-b5ba-1768306ddb74"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:03:11.202929 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.202795    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-token\" (UniqueName: \"kubernetes.io/projected/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-token\") pod \"df6ef068-9dda-4168-b5ba-1768306ddb74\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") "
Nov 02 02:03:11.202929 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.202879    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-workload-socket\") pod \"df6ef068-9dda-4168-b5ba-1768306ddb74\" (UID: \"df6ef068-9dda-4168-b5ba-1768306ddb74\") "
Nov 02 02:03:11.202929 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.202893    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-data" pod "df6ef068-9dda-4168-b5ba-1768306ddb74" (UID: "df6ef068-9dda-4168-b5ba-1768306ddb74"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:03:11.203151 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.202927    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-credential-socket" pod "df6ef068-9dda-4168-b5ba-1768306ddb74" (UID: "df6ef068-9dda-4168-b5ba-1768306ddb74"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:03:11.203151 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.203047    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df6ef068-9dda-4168-b5ba-1768306ddb74-istiod-ca-cert" pod "df6ef068-9dda-4168-b5ba-1768306ddb74" (UID: "df6ef068-9dda-4168-b5ba-1768306ddb74"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:03:11.203263 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.203230    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-workload-socket" pod "df6ef068-9dda-4168-b5ba-1768306ddb74" (UID: "df6ef068-9dda-4168-b5ba-1768306ddb74"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:03:11.203698 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.203671    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:11.203831 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.203703    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:11.203831 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.203719    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df6ef068-9dda-4168-b5ba-1768306ddb74-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:11.203831 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.203733    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:11.203831 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.203748    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:11.204766 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.204741    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df6ef068-9dda-4168-b5ba-1768306ddb74-kube-api-access-b2qps" pod "df6ef068-9dda-4168-b5ba-1768306ddb74" (UID: "df6ef068-9dda-4168-b5ba-1768306ddb74"). InnerVolumeSpecName "kube-api-access-b2qps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:03:11.204896 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.204862    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-envoy" pod "df6ef068-9dda-4168-b5ba-1768306ddb74" (UID: "df6ef068-9dda-4168-b5ba-1768306ddb74"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:03:11.204896 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.204877    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-podinfo" pod "df6ef068-9dda-4168-b5ba-1768306ddb74" (UID: "df6ef068-9dda-4168-b5ba-1768306ddb74"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:03:11.205092 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.205073    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-token" pod "df6ef068-9dda-4168-b5ba-1768306ddb74" (UID: "df6ef068-9dda-4168-b5ba-1768306ddb74"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:03:11.304979 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.304947    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2qps\" (UniqueName: \"kubernetes.io/projected/df6ef068-9dda-4168-b5ba-1768306ddb74-kube-api-access-b2qps\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:11.304979 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.304977    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:11.305197 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.304991    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:11.305197 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:11.305005    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df6ef068-9dda-4168-b5ba-1768306ddb74-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:12.050646 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:12.050615    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"
Nov 02 02:03:12.051098 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:12.050620    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk" event={"ID":"df6ef068-9dda-4168-b5ba-1768306ddb74","Type":"ContainerDied","Data":"ddd99b89baf8e0d626ad6dffb1b6da610f2c8f91efe4f43f9090ebe8f0932691"}
Nov 02 02:03:12.051098 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:12.050744    4515 scope.go:122] "RemoveContainer" containerID="2720db1774a79797afc107bf72b6a9127aeecd8a60a82b54858f74875e668734"
Nov 02 02:03:12.271601 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:12.271568    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"]
Nov 02 02:03:12.274305 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:12.274284    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-ss2fk"]
Nov 02 02:03:12.954652 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:12.954614    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df6ef068-9dda-4168-b5ba-1768306ddb74" path="/var/lib/kubelet/pods/df6ef068-9dda-4168-b5ba-1768306ddb74/volumes"
Nov 02 02:03:13.869100 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:13.869059    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"]
Nov 02 02:03:13.869555 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:13.869294    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn" podUID="59aa703a-473b-4038-a9d8-6ae2e618f010" containerName="istio-proxy" containerID="cri-o://39a082441cda730a122f3d3933b915afe4bf61871bc6ce712417426c7f31bf4a" gracePeriod=30
Nov 02 02:03:18.132228 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:18.132190    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 02:03:18.132650 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:18.132266    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn" podUID="59aa703a-473b-4038-a9d8-6ae2e618f010" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 02:03:19.145191 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.144378    4515 generic.go:356] "Generic (PLEG): container finished" podID="59aa703a-473b-4038-a9d8-6ae2e618f010" containerID="39a082441cda730a122f3d3933b915afe4bf61871bc6ce712417426c7f31bf4a" exitCode=0
Nov 02 02:03:19.145191 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.144448    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn" event={"ID":"59aa703a-473b-4038-a9d8-6ae2e618f010","Type":"ContainerDied","Data":"39a082441cda730a122f3d3933b915afe4bf61871bc6ce712417426c7f31bf4a"}
Nov 02 02:03:19.188938 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.188914    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 02:03:19.241306 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.241273    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-workload-socket\") pod \"59aa703a-473b-4038-a9d8-6ae2e618f010\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") "
Nov 02 02:03:19.241306 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.241310    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-podinfo\") pod \"59aa703a-473b-4038-a9d8-6ae2e618f010\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") "
Nov 02 02:03:19.241563 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.241338    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-token\" (UniqueName: \"kubernetes.io/projected/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-token\") pod \"59aa703a-473b-4038-a9d8-6ae2e618f010\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") "
Nov 02 02:03:19.241563 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.241441    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/59aa703a-473b-4038-a9d8-6ae2e618f010-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59aa703a-473b-4038-a9d8-6ae2e618f010-istiod-ca-cert\") pod \"59aa703a-473b-4038-a9d8-6ae2e618f010\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") "
Nov 02 02:03:19.241563 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.241514    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-data\") pod \"59aa703a-473b-4038-a9d8-6ae2e618f010\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") "
Nov 02 02:03:19.241764 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.241591    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-workload-socket" pod "59aa703a-473b-4038-a9d8-6ae2e618f010" (UID: "59aa703a-473b-4038-a9d8-6ae2e618f010"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:03:19.241764 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.241605    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59aa703a-473b-4038-a9d8-6ae2e618f010-kube-api-access-m4lhq\" (UniqueName: \"kubernetes.io/projected/59aa703a-473b-4038-a9d8-6ae2e618f010-kube-api-access-m4lhq\") pod \"59aa703a-473b-4038-a9d8-6ae2e618f010\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") "
Nov 02 02:03:19.241764 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.241727    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-envoy\") pod \"59aa703a-473b-4038-a9d8-6ae2e618f010\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") "
Nov 02 02:03:19.241952 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.241769    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-workload-certs\") pod \"59aa703a-473b-4038-a9d8-6ae2e618f010\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") "
Nov 02 02:03:19.241952 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.241786    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-data" pod "59aa703a-473b-4038-a9d8-6ae2e618f010" (UID: "59aa703a-473b-4038-a9d8-6ae2e618f010"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:03:19.241952 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.241818    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-credential-socket\") pod \"59aa703a-473b-4038-a9d8-6ae2e618f010\" (UID: \"59aa703a-473b-4038-a9d8-6ae2e618f010\") "
Nov 02 02:03:19.241952 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.241852    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59aa703a-473b-4038-a9d8-6ae2e618f010-istiod-ca-cert" pod "59aa703a-473b-4038-a9d8-6ae2e618f010" (UID: "59aa703a-473b-4038-a9d8-6ae2e618f010"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:03:19.242202 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.241995    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-workload-certs" pod "59aa703a-473b-4038-a9d8-6ae2e618f010" (UID: "59aa703a-473b-4038-a9d8-6ae2e618f010"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:03:19.242202 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.242089    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-credential-socket" pod "59aa703a-473b-4038-a9d8-6ae2e618f010" (UID: "59aa703a-473b-4038-a9d8-6ae2e618f010"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:03:19.242577 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.242547    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59aa703a-473b-4038-a9d8-6ae2e618f010-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:19.242703 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.242579    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:19.242703 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.242596    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:19.242703 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.242612    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:19.242703 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.242627    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:19.243936 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.243909    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59aa703a-473b-4038-a9d8-6ae2e618f010-kube-api-access-m4lhq" pod "59aa703a-473b-4038-a9d8-6ae2e618f010" (UID: "59aa703a-473b-4038-a9d8-6ae2e618f010"). InnerVolumeSpecName "kube-api-access-m4lhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:03:19.244044 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.243997    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-token" pod "59aa703a-473b-4038-a9d8-6ae2e618f010" (UID: "59aa703a-473b-4038-a9d8-6ae2e618f010"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:03:19.244044 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.244012    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-envoy" pod "59aa703a-473b-4038-a9d8-6ae2e618f010" (UID: "59aa703a-473b-4038-a9d8-6ae2e618f010"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:03:19.244141 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.244054    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-podinfo" pod "59aa703a-473b-4038-a9d8-6ae2e618f010" (UID: "59aa703a-473b-4038-a9d8-6ae2e618f010"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:03:19.344212 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.344177    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:19.344212 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.344207    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:19.344212 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.344218    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4lhq\" (UniqueName: \"kubernetes.io/projected/59aa703a-473b-4038-a9d8-6ae2e618f010-kube-api-access-m4lhq\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:19.344452 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:19.344230    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59aa703a-473b-4038-a9d8-6ae2e618f010-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:03:20.151509 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:20.151479    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"
Nov 02 02:03:20.151914 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:20.151510    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn" event={"ID":"59aa703a-473b-4038-a9d8-6ae2e618f010","Type":"ContainerDied","Data":"23ff6f4d7e1c735de01b544d00a05acad7b36531207c7e7df84c25870117985a"}
Nov 02 02:03:20.151914 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:20.151578    4515 scope.go:122] "RemoveContainer" containerID="39a082441cda730a122f3d3933b915afe4bf61871bc6ce712417426c7f31bf4a"
Nov 02 02:03:20.368187 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:20.368156    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"]
Nov 02 02:03:20.417982 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:20.417909    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2g5sn"]
Nov 02 02:03:20.955600 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:03:20.955567    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="59aa703a-473b-4038-a9d8-6ae2e618f010" path="/var/lib/kubelet/pods/59aa703a-473b-4038-a9d8-6ae2e618f010/volumes"
Nov 02 02:03:39.960683 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:03:39.960642    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:04:09.959210 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:04:09.959131    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:04:39.960191 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:04:39.960147    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:04:59.495755 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.495718    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7028-1f4df2"]
Nov 02 02:04:59.496312 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.496296    4515 cpu_manager.go:414] "containerMap: removing container" podUID="a440be2e-a939-4a5d-a046-93083c782eb0" containerName="cleanup"
Nov 02 02:04:59.496367 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.496315    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a440be2e-a939-4a5d-a046-93083c782eb0" containerName="cleanup"
Nov 02 02:04:59.496367 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.496327    4515 cpu_manager.go:414] "containerMap: removing container" podUID="df6ef068-9dda-4168-b5ba-1768306ddb74" containerName="istio-proxy"
Nov 02 02:04:59.496367 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.496331    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df6ef068-9dda-4168-b5ba-1768306ddb74" containerName="istio-proxy"
Nov 02 02:04:59.496367 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.496358    4515 cpu_manager.go:414] "containerMap: removing container" podUID="59aa703a-473b-4038-a9d8-6ae2e618f010" containerName="istio-proxy"
Nov 02 02:04:59.496367 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.496364    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59aa703a-473b-4038-a9d8-6ae2e618f010" containerName="istio-proxy"
Nov 02 02:04:59.496521 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.496436    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="a440be2e-a939-4a5d-a046-93083c782eb0" containerName="cleanup"
Nov 02 02:04:59.496521 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.496449    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c9faeb1-598c-43e1-bbf0-4907010bf722" containerName="cleanup"
Nov 02 02:04:59.496521 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.496456    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="59aa703a-473b-4038-a9d8-6ae2e618f010" containerName="istio-proxy"
Nov 02 02:04:59.496521 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.496465    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="df6ef068-9dda-4168-b5ba-1768306ddb74" containerName="istio-proxy"
Nov 02 02:04:59.499359 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.499341    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7028-1f4df2"
Nov 02 02:04:59.563062 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.551579    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-v4n9t\""
Nov 02 02:04:59.563062 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.556139    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7028-1f4df2"]
Nov 02 02:04:59.627038 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.627008    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g48zw\" (UniqueName: \"kubernetes.io/projected/4d8a0143-8900-44c1-aa7c-bf3b8503a938-kube-api-access-g48zw\") pod \"test-curl-7028-1f4df2\" (UID: \"4d8a0143-8900-44c1-aa7c-bf3b8503a938\") " pod="opendatahub/test-curl-7028-1f4df2"
Nov 02 02:04:59.728932 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.728902    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g48zw\" (UniqueName: \"kubernetes.io/projected/4d8a0143-8900-44c1-aa7c-bf3b8503a938-kube-api-access-g48zw\") pod \"test-curl-7028-1f4df2\" (UID: \"4d8a0143-8900-44c1-aa7c-bf3b8503a938\") " pod="opendatahub/test-curl-7028-1f4df2"
Nov 02 02:04:59.789318 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.789245    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g48zw\" (UniqueName: \"kubernetes.io/projected/4d8a0143-8900-44c1-aa7c-bf3b8503a938-kube-api-access-g48zw\") pod \"test-curl-7028-1f4df2\" (UID: \"4d8a0143-8900-44c1-aa7c-bf3b8503a938\") " pod="opendatahub/test-curl-7028-1f4df2"
Nov 02 02:04:59.811028 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:04:59.811001    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7028-1f4df2"
Nov 02 02:05:00.191882 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:00.191858    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7028-1f4df2"]
Nov 02 02:05:00.193760 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:05:00.193730    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d8a0143_8900_44c1_aa7c_bf3b8503a938.slice/crio-6f053fa69a5ae0ee90028514903867ae09ea99f2efde0b327de63541606467d3 WatchSource:0}: Error finding container 6f053fa69a5ae0ee90028514903867ae09ea99f2efde0b327de63541606467d3: Status 404 returned error can't find the container with id 6f053fa69a5ae0ee90028514903867ae09ea99f2efde0b327de63541606467d3
Nov 02 02:05:00.773982 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:00.773947    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7028-1f4df2" event={"ID":"4d8a0143-8900-44c1-aa7c-bf3b8503a938","Type":"ContainerStarted","Data":"6f053fa69a5ae0ee90028514903867ae09ea99f2efde0b327de63541606467d3"}
Nov 02 02:05:01.781362 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:01.781326    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7028-1f4df2" event={"ID":"4d8a0143-8900-44c1-aa7c-bf3b8503a938","Type":"ContainerStarted","Data":"5188c6b53c5c33af422472671bdbc5497ae804f599f8ed31415d3e54d97abad7"}
Nov 02 02:05:01.902597 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:01.902528    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7028-1f4df2" podStartSLOduration=1.7741535819999998 podStartE2EDuration="2.902498199s" totalImagesPullingTime="1.128344617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:05:01.900311998 +0000 UTC m=+1991.112681808" watchObservedRunningTime="2026-11-02 02:05:01.902498199 +0000 UTC m=+1991.114868005"
Nov 02 02:05:03.429286 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:03.429241    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7028-1f4df2"]
Nov 02 02:05:03.432732 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:03.432696    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7028-1f4df2"]
Nov 02 02:05:03.792879 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:03.792823    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7028-1f4df2" podUID="4d8a0143-8900-44c1-aa7c-bf3b8503a938" containerName="test-curl-7028-1f4df2" containerID="cri-o://5188c6b53c5c33af422472671bdbc5497ae804f599f8ed31415d3e54d97abad7" gracePeriod=2
Nov 02 02:05:03.843767 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:05:03.843725    4515 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7028-1f4df2\" is forbidden: User \"system:node:ip-10-0-138-75.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-75.us-west-1.compute.internal' and this object" podUID="4d8a0143-8900-44c1-aa7c-bf3b8503a938" pod="opendatahub/test-curl-7028-1f4df2"
Nov 02 02:05:06.024523 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:06.024496    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7028-1f4df2"
Nov 02 02:05:06.138746 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:06.138659    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d8a0143-8900-44c1-aa7c-bf3b8503a938-kube-api-access-g48zw\" (UniqueName: \"kubernetes.io/projected/4d8a0143-8900-44c1-aa7c-bf3b8503a938-kube-api-access-g48zw\") pod \"4d8a0143-8900-44c1-aa7c-bf3b8503a938\" (UID: \"4d8a0143-8900-44c1-aa7c-bf3b8503a938\") "
Nov 02 02:05:06.144898 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:06.144844    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d8a0143-8900-44c1-aa7c-bf3b8503a938-kube-api-access-g48zw" pod "4d8a0143-8900-44c1-aa7c-bf3b8503a938" (UID: "4d8a0143-8900-44c1-aa7c-bf3b8503a938"). InnerVolumeSpecName "kube-api-access-g48zw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:05:06.241221 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:06.241186    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g48zw\" (UniqueName: \"kubernetes.io/projected/4d8a0143-8900-44c1-aa7c-bf3b8503a938-kube-api-access-g48zw\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:05:06.810415 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:06.810380    4515 generic.go:356] "Generic (PLEG): container finished" podID="4d8a0143-8900-44c1-aa7c-bf3b8503a938" containerID="5188c6b53c5c33af422472671bdbc5497ae804f599f8ed31415d3e54d97abad7" exitCode=137
Nov 02 02:05:06.810609 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:06.810432    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7028-1f4df2"
Nov 02 02:05:06.810609 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:06.810474    4515 scope.go:122] "RemoveContainer" containerID="5188c6b53c5c33af422472671bdbc5497ae804f599f8ed31415d3e54d97abad7"
Nov 02 02:05:06.820832 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:06.820814    4515 scope.go:122] "RemoveContainer" containerID="5188c6b53c5c33af422472671bdbc5497ae804f599f8ed31415d3e54d97abad7"
Nov 02 02:05:06.821127 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:05:06.821092    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5188c6b53c5c33af422472671bdbc5497ae804f599f8ed31415d3e54d97abad7\": container with ID starting with 5188c6b53c5c33af422472671bdbc5497ae804f599f8ed31415d3e54d97abad7 not found: ID does not exist" containerID="5188c6b53c5c33af422472671bdbc5497ae804f599f8ed31415d3e54d97abad7"
Nov 02 02:05:06.821201 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:06.821140    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5188c6b53c5c33af422472671bdbc5497ae804f599f8ed31415d3e54d97abad7"} err="failed to get container status \"5188c6b53c5c33af422472671bdbc5497ae804f599f8ed31415d3e54d97abad7\": rpc error: code = NotFound desc = could not find container \"5188c6b53c5c33af422472671bdbc5497ae804f599f8ed31415d3e54d97abad7\": container with ID starting with 5188c6b53c5c33af422472671bdbc5497ae804f599f8ed31415d3e54d97abad7 not found: ID does not exist"
Nov 02 02:05:06.869893 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:05:06.869866    4515 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7028-1f4df2\" is forbidden: User \"system:node:ip-10-0-138-75.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-75.us-west-1.compute.internal' and this object" podUID="4d8a0143-8900-44c1-aa7c-bf3b8503a938" pod="opendatahub/test-curl-7028-1f4df2"
Nov 02 02:05:06.953294 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:06.953266    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d8a0143-8900-44c1-aa7c-bf3b8503a938" path="/var/lib/kubelet/pods/4d8a0143-8900-44c1-aa7c-bf3b8503a938/volumes"
Nov 02 02:05:09.959068 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:05:09.959036    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:05:29.959245 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:29.959212    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"]
Nov 02 02:05:29.961773 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:29.959406    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np" podUID="47069c5d-070f-425b-ba94-a69d0585fd29" containerName="istio-proxy" containerID="cri-o://39e46d97ddf088ca52e1e2bfa0ffdcf7a9daa2208b005b62bd1b3618aa3f5db1" gracePeriod=30
Nov 02 02:05:35.236069 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.236002    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 02:05:35.286978 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.286949    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/47069c5d-070f-425b-ba94-a69d0585fd29-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47069c5d-070f-425b-ba94-a69d0585fd29-istiod-ca-cert\") pod \"47069c5d-070f-425b-ba94-a69d0585fd29\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") "
Nov 02 02:05:35.287159 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.286997    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47069c5d-070f-425b-ba94-a69d0585fd29-istio-token\" (UniqueName: \"kubernetes.io/projected/47069c5d-070f-425b-ba94-a69d0585fd29-istio-token\") pod \"47069c5d-070f-425b-ba94-a69d0585fd29\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") "
Nov 02 02:05:35.287159 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.287022    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-workload-socket\") pod \"47069c5d-070f-425b-ba94-a69d0585fd29\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") "
Nov 02 02:05:35.287159 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.287045    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-istio-envoy\") pod \"47069c5d-070f-425b-ba94-a69d0585fd29\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") "
Nov 02 02:05:35.287289 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.287205    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-credential-socket\") pod \"47069c5d-070f-425b-ba94-a69d0585fd29\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") "
Nov 02 02:05:35.287289 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.287261    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-istio-data\") pod \"47069c5d-070f-425b-ba94-a69d0585fd29\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") "
Nov 02 02:05:35.287396 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.287299    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-workload-certs\") pod \"47069c5d-070f-425b-ba94-a69d0585fd29\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") "
Nov 02 02:05:35.287459 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.287412    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/47069c5d-070f-425b-ba94-a69d0585fd29-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47069c5d-070f-425b-ba94-a69d0585fd29-istio-podinfo\") pod \"47069c5d-070f-425b-ba94-a69d0585fd29\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") "
Nov 02 02:05:35.287539 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.287512    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47069c5d-070f-425b-ba94-a69d0585fd29-kube-api-access-6btzh\" (UniqueName: \"kubernetes.io/projected/47069c5d-070f-425b-ba94-a69d0585fd29-kube-api-access-6btzh\") pod \"47069c5d-070f-425b-ba94-a69d0585fd29\" (UID: \"47069c5d-070f-425b-ba94-a69d0585fd29\") "
Nov 02 02:05:35.287610 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.287407    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47069c5d-070f-425b-ba94-a69d0585fd29-istiod-ca-cert" pod "47069c5d-070f-425b-ba94-a69d0585fd29" (UID: "47069c5d-070f-425b-ba94-a69d0585fd29"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:05:35.287671 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.287410    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-workload-socket" pod "47069c5d-070f-425b-ba94-a69d0585fd29" (UID: "47069c5d-070f-425b-ba94-a69d0585fd29"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:05:35.287671 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.287574    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-credential-socket" pod "47069c5d-070f-425b-ba94-a69d0585fd29" (UID: "47069c5d-070f-425b-ba94-a69d0585fd29"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:05:35.288070 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.288022    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-istio-data" pod "47069c5d-070f-425b-ba94-a69d0585fd29" (UID: "47069c5d-070f-425b-ba94-a69d0585fd29"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:05:35.288070 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.288053    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-workload-certs" pod "47069c5d-070f-425b-ba94-a69d0585fd29" (UID: "47069c5d-070f-425b-ba94-a69d0585fd29"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:05:35.288311 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.288257    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47069c5d-070f-425b-ba94-a69d0585fd29-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:05:35.288311 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.288281    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:05:35.288311 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.288296    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:05:35.288311 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.288309    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:05:35.288519 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.288323    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:05:35.289452 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.289419    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-istio-envoy" pod "47069c5d-070f-425b-ba94-a69d0585fd29" (UID: "47069c5d-070f-425b-ba94-a69d0585fd29"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:05:35.289538 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.289475    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47069c5d-070f-425b-ba94-a69d0585fd29-istio-token" pod "47069c5d-070f-425b-ba94-a69d0585fd29" (UID: "47069c5d-070f-425b-ba94-a69d0585fd29"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:05:35.289723 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.289707    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/47069c5d-070f-425b-ba94-a69d0585fd29-istio-podinfo" pod "47069c5d-070f-425b-ba94-a69d0585fd29" (UID: "47069c5d-070f-425b-ba94-a69d0585fd29"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:05:35.290023 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.289999    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47069c5d-070f-425b-ba94-a69d0585fd29-kube-api-access-6btzh" pod "47069c5d-070f-425b-ba94-a69d0585fd29" (UID: "47069c5d-070f-425b-ba94-a69d0585fd29"). InnerVolumeSpecName "kube-api-access-6btzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:05:35.389331 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.389302    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47069c5d-070f-425b-ba94-a69d0585fd29-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:05:35.389331 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.389329    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6btzh\" (UniqueName: \"kubernetes.io/projected/47069c5d-070f-425b-ba94-a69d0585fd29-kube-api-access-6btzh\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:05:35.389532 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.389340    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47069c5d-070f-425b-ba94-a69d0585fd29-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:05:35.389532 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:35.389349    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47069c5d-070f-425b-ba94-a69d0585fd29-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:05:36.032946 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:36.032909    4515 generic.go:356] "Generic (PLEG): container finished" podID="47069c5d-070f-425b-ba94-a69d0585fd29" containerID="39e46d97ddf088ca52e1e2bfa0ffdcf7a9daa2208b005b62bd1b3618aa3f5db1" exitCode=0
Nov 02 02:05:36.033160 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:36.032971    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"
Nov 02 02:05:36.033160 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:36.032997    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np" event={"ID":"47069c5d-070f-425b-ba94-a69d0585fd29","Type":"ContainerDied","Data":"39e46d97ddf088ca52e1e2bfa0ffdcf7a9daa2208b005b62bd1b3618aa3f5db1"}
Nov 02 02:05:36.033160 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:36.033066    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np" event={"ID":"47069c5d-070f-425b-ba94-a69d0585fd29","Type":"ContainerDied","Data":"7b8701b5f13b32baa32ddd5fc983de123f61c44a6f7e64992c291547b33fbfa1"}
Nov 02 02:05:36.033160 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:36.033087    4515 scope.go:122] "RemoveContainer" containerID="39e46d97ddf088ca52e1e2bfa0ffdcf7a9daa2208b005b62bd1b3618aa3f5db1"
Nov 02 02:05:36.043224 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:36.043195    4515 scope.go:122] "RemoveContainer" containerID="39e46d97ddf088ca52e1e2bfa0ffdcf7a9daa2208b005b62bd1b3618aa3f5db1"
Nov 02 02:05:36.043556 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:05:36.043515    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39e46d97ddf088ca52e1e2bfa0ffdcf7a9daa2208b005b62bd1b3618aa3f5db1\": container with ID starting with 39e46d97ddf088ca52e1e2bfa0ffdcf7a9daa2208b005b62bd1b3618aa3f5db1 not found: ID does not exist" containerID="39e46d97ddf088ca52e1e2bfa0ffdcf7a9daa2208b005b62bd1b3618aa3f5db1"
Nov 02 02:05:36.043652 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:36.043546    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39e46d97ddf088ca52e1e2bfa0ffdcf7a9daa2208b005b62bd1b3618aa3f5db1"} err="failed to get container status \"39e46d97ddf088ca52e1e2bfa0ffdcf7a9daa2208b005b62bd1b3618aa3f5db1\": rpc error: code = NotFound desc = could not find container \"39e46d97ddf088ca52e1e2bfa0ffdcf7a9daa2208b005b62bd1b3618aa3f5db1\": container with ID starting with 39e46d97ddf088ca52e1e2bfa0ffdcf7a9daa2208b005b62bd1b3618aa3f5db1 not found: ID does not exist"
Nov 02 02:05:36.247048 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:36.247010    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"]
Nov 02 02:05:36.296595 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:36.296525    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-2v9np"]
Nov 02 02:05:36.954057 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:36.954022    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="47069c5d-070f-425b-ba94-a69d0585fd29" path="/var/lib/kubelet/pods/47069c5d-070f-425b-ba94-a69d0585fd29/volumes"
Nov 02 02:05:40.038147 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:05:40.036395    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:05:58.975296 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:58.975262    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"]
Nov 02 02:05:58.975872 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:58.975522    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw" podUID="3a2f9447-a086-4541-8a70-4e6dc5b98ac0" containerName="istio-proxy" containerID="cri-o://673c3195b20a388e0613bfb62afcd0ba68c6994082fdefebbcf3a8acdf101068" gracePeriod=30
Nov 02 02:05:59.000858 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:59.000827    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"]
Nov 02 02:05:59.001057 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:05:59.001021    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x" podUID="3f4a20c6-793d-44fd-988e-a1b950a71ea7" containerName="istio-proxy" containerID="cri-o://b88e57c88ba90343101c19464164375d9ef549fc58c1bd6d5f765a8f7c5bd9f4" gracePeriod=30
Nov 02 02:06:00.595631 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:00.595595    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"]
Nov 02 02:06:00.596164 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:00.595833    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4" podUID="90baf3f1-927b-44b1-b63b-3f6503b4fee3" containerName="istio-proxy" containerID="cri-o://dd411abacd787178ba930ac6ef5a43f602dd22456d7b8587607e7889d0f8f4cb" gracePeriod=30
Nov 02 02:06:00.646840 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:00.646796    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"]
Nov 02 02:06:00.647078 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:00.647036    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx" podUID="444174f7-6dd7-411d-b625-e3edd0f214ce" containerName="istio-proxy" containerID="cri-o://67d599c809cf9b757f0ab84bf60b9d4172ba11a6071d9fddae72a551bfad3c18" gracePeriod=30
Nov 02 02:06:01.519678 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:01.519642    4515 patch_prober.go:29] interesting pod/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 02:06:01.519856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:01.519708    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x" podUID="3f4a20c6-793d-44fd-988e-a1b950a71ea7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 02:06:02.998031 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:02.997994    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 02:06:02.998457 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:02.998061    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4" podUID="90baf3f1-927b-44b1-b63b-3f6503b4fee3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 02:06:04.239828 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.239753    4515 generic.go:356] "Generic (PLEG): container finished" podID="3f4a20c6-793d-44fd-988e-a1b950a71ea7" containerID="b88e57c88ba90343101c19464164375d9ef549fc58c1bd6d5f765a8f7c5bd9f4" exitCode=0
Nov 02 02:06:04.240278 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.239902    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x" event={"ID":"3f4a20c6-793d-44fd-988e-a1b950a71ea7","Type":"ContainerDied","Data":"b88e57c88ba90343101c19464164375d9ef549fc58c1bd6d5f765a8f7c5bd9f4"}
Nov 02 02:06:04.242200 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.242174    4515 generic.go:356] "Generic (PLEG): container finished" podID="3a2f9447-a086-4541-8a70-4e6dc5b98ac0" containerID="673c3195b20a388e0613bfb62afcd0ba68c6994082fdefebbcf3a8acdf101068" exitCode=0
Nov 02 02:06:04.242300 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.242248    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw" event={"ID":"3a2f9447-a086-4541-8a70-4e6dc5b98ac0","Type":"ContainerDied","Data":"673c3195b20a388e0613bfb62afcd0ba68c6994082fdefebbcf3a8acdf101068"}
Nov 02 02:06:04.290059 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.290033    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 02:06:04.307778 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.307756    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 02:06:04.439672 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.439639    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-data\") pod \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") "
Nov 02 02:06:04.439836 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.439695    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-token\" (UniqueName: \"kubernetes.io/projected/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-token\") pod \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") "
Nov 02 02:06:04.439836 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.439729    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-token\" (UniqueName: \"kubernetes.io/projected/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-token\") pod \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") "
Nov 02 02:06:04.439836 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.439757    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-workload-socket\") pod \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") "
Nov 02 02:06:04.439836 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.439786    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-data\") pod \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") "
Nov 02 02:06:04.439836 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.439831    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-envoy\") pod \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") "
Nov 02 02:06:04.440103 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.439900    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-workload-certs\") pod \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") "
Nov 02 02:06:04.440103 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.439926    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-data" pod "3a2f9447-a086-4541-8a70-4e6dc5b98ac0" (UID: "3a2f9447-a086-4541-8a70-4e6dc5b98ac0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:04.440103 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.439996    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-workload-certs\") pod \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") "
Nov 02 02:06:04.440103 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440050    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-envoy\") pod \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") "
Nov 02 02:06:04.440103 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440086    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-credential-socket\") pod \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") "
Nov 02 02:06:04.440103 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440055    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-workload-socket" pod "3a2f9447-a086-4541-8a70-4e6dc5b98ac0" (UID: "3a2f9447-a086-4541-8a70-4e6dc5b98ac0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:04.440441 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440134    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-data" pod "3f4a20c6-793d-44fd-988e-a1b950a71ea7" (UID: "3f4a20c6-793d-44fd-988e-a1b950a71ea7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:04.440441 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440111    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istiod-ca-cert\") pod \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") "
Nov 02 02:06:04.440441 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440191    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-workload-certs" pod "3f4a20c6-793d-44fd-988e-a1b950a71ea7" (UID: "3f4a20c6-793d-44fd-988e-a1b950a71ea7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:04.440441 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440237    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-workload-socket\") pod \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") "
Nov 02 02:06:04.440441 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440269    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-kube-api-access-v8nhs\" (UniqueName: \"kubernetes.io/projected/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-kube-api-access-v8nhs\") pod \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") "
Nov 02 02:06:04.440441 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440308    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-credential-socket" pod "3a2f9447-a086-4541-8a70-4e6dc5b98ac0" (UID: "3a2f9447-a086-4541-8a70-4e6dc5b98ac0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:04.440441 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440323    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-podinfo\") pod \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") "
Nov 02 02:06:04.440441 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440369    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-podinfo\") pod \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\" (UID: \"3a2f9447-a086-4541-8a70-4e6dc5b98ac0\") "
Nov 02 02:06:04.440441 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440385    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-workload-certs" pod "3a2f9447-a086-4541-8a70-4e6dc5b98ac0" (UID: "3a2f9447-a086-4541-8a70-4e6dc5b98ac0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:04.440441 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440410    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istiod-ca-cert\") pod \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") "
Nov 02 02:06:04.440441 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440434    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istiod-ca-cert" pod "3a2f9447-a086-4541-8a70-4e6dc5b98ac0" (UID: "3a2f9447-a086-4541-8a70-4e6dc5b98ac0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:06:04.441026 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440451    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-credential-socket\") pod \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") "
Nov 02 02:06:04.441026 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440490    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f4a20c6-793d-44fd-988e-a1b950a71ea7-kube-api-access-4b6m9\" (UniqueName: \"kubernetes.io/projected/3f4a20c6-793d-44fd-988e-a1b950a71ea7-kube-api-access-4b6m9\") pod \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\" (UID: \"3f4a20c6-793d-44fd-988e-a1b950a71ea7\") "
Nov 02 02:06:04.441026 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440532    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-workload-socket" pod "3f4a20c6-793d-44fd-988e-a1b950a71ea7" (UID: "3f4a20c6-793d-44fd-988e-a1b950a71ea7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:04.441026 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.440839    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istiod-ca-cert" pod "3f4a20c6-793d-44fd-988e-a1b950a71ea7" (UID: "3f4a20c6-793d-44fd-988e-a1b950a71ea7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:06:04.441285 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.441261    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.441351 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.441290    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.441351 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.441304    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.441351 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.441318    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.441351 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.441333    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.441351 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.441346    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.441601 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.441360    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.441601 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.441374    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.441601 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.441388    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.441601 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.441579    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-credential-socket" pod "3f4a20c6-793d-44fd-988e-a1b950a71ea7" (UID: "3f4a20c6-793d-44fd-988e-a1b950a71ea7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:04.442571 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.442544    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-token" pod "3a2f9447-a086-4541-8a70-4e6dc5b98ac0" (UID: "3a2f9447-a086-4541-8a70-4e6dc5b98ac0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:06:04.442715 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.442676    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-token" pod "3f4a20c6-793d-44fd-988e-a1b950a71ea7" (UID: "3f4a20c6-793d-44fd-988e-a1b950a71ea7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:06:04.442844 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.442817    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-kube-api-access-v8nhs" pod "3a2f9447-a086-4541-8a70-4e6dc5b98ac0" (UID: "3a2f9447-a086-4541-8a70-4e6dc5b98ac0"). InnerVolumeSpecName "kube-api-access-v8nhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:06:04.443084 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.443056    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-podinfo" pod "3a2f9447-a086-4541-8a70-4e6dc5b98ac0" (UID: "3a2f9447-a086-4541-8a70-4e6dc5b98ac0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:06:04.443357 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.443334    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f4a20c6-793d-44fd-988e-a1b950a71ea7-kube-api-access-4b6m9" pod "3f4a20c6-793d-44fd-988e-a1b950a71ea7" (UID: "3f4a20c6-793d-44fd-988e-a1b950a71ea7"). InnerVolumeSpecName "kube-api-access-4b6m9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:06:04.443442 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.443389    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-envoy" pod "3a2f9447-a086-4541-8a70-4e6dc5b98ac0" (UID: "3a2f9447-a086-4541-8a70-4e6dc5b98ac0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:04.443579 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.443566    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-envoy" pod "3f4a20c6-793d-44fd-988e-a1b950a71ea7" (UID: "3f4a20c6-793d-44fd-988e-a1b950a71ea7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:04.443884 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.443869    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-podinfo" pod "3f4a20c6-793d-44fd-988e-a1b950a71ea7" (UID: "3f4a20c6-793d-44fd-988e-a1b950a71ea7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:06:04.543278 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.543198    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.543278 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.543226    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.543278 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.543235    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.543278 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.543243    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.543278 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.543252    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8nhs\" (UniqueName: \"kubernetes.io/projected/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-kube-api-access-v8nhs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.543278 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.543261    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f4a20c6-793d-44fd-988e-a1b950a71ea7-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.543278 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.543270    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a2f9447-a086-4541-8a70-4e6dc5b98ac0-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.543278 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.543280    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f4a20c6-793d-44fd-988e-a1b950a71ea7-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.543633 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.543290    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4b6m9\" (UniqueName: \"kubernetes.io/projected/3f4a20c6-793d-44fd-988e-a1b950a71ea7-kube-api-access-4b6m9\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:04.744043 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.744008    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"]
Nov 02 02:06:04.744256 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:04.744232    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg" podUID="dbce4552-66b5-4a8f-ad14-edacdc87ea52" containerName="istio-proxy" containerID="cri-o://a78258ff2753e157bc6fcf46d270506c66560411ef315e5ecb5e6d335aaf5ab4" gracePeriod=30
Nov 02 02:06:05.250637 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:05.250549    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"
Nov 02 02:06:05.250637 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:05.250572    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x" event={"ID":"3f4a20c6-793d-44fd-988e-a1b950a71ea7","Type":"ContainerDied","Data":"4faa227cf1b68f41ad7bb0197f26c9e3a08dad05016e3e864b4b491639ccabfe"}
Nov 02 02:06:05.251201 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:05.250646    4515 scope.go:122] "RemoveContainer" containerID="b88e57c88ba90343101c19464164375d9ef549fc58c1bd6d5f765a8f7c5bd9f4"
Nov 02 02:06:05.252169 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:05.252147    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw" event={"ID":"3a2f9447-a086-4541-8a70-4e6dc5b98ac0","Type":"ContainerDied","Data":"ea3680026f45bdbe8b13766d50d489d1a4b875c45544c1942ddd07a23a3be9f5"}
Nov 02 02:06:05.252251 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:05.252238    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"
Nov 02 02:06:05.262630 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:05.262605    4515 scope.go:122] "RemoveContainer" containerID="673c3195b20a388e0613bfb62afcd0ba68c6994082fdefebbcf3a8acdf101068"
Nov 02 02:06:05.415050 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:05.415016    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"]
Nov 02 02:06:05.465042 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:05.465003    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-x5v8x"]
Nov 02 02:06:05.575215 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:05.575131    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"]
Nov 02 02:06:05.580793 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:05.580766    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-pkmtw"]
Nov 02 02:06:05.919332 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:05.919305    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 02:06:05.939625 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:05.939598    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 02:06:06.061391 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.061355    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-workload-socket\") pod \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") "
Nov 02 02:06:06.061571 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.061484    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-data\") pod \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") "
Nov 02 02:06:06.061571 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.061530    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-token\" (UniqueName: \"kubernetes.io/projected/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-token\") pod \"444174f7-6dd7-411d-b625-e3edd0f214ce\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") "
Nov 02 02:06:06.061667 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.061578    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-token\" (UniqueName: \"kubernetes.io/projected/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-token\") pod \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") "
Nov 02 02:06:06.061667 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.061608    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90baf3f1-927b-44b1-b63b-3f6503b4fee3-kube-api-access-qb8fk\" (UniqueName: \"kubernetes.io/projected/90baf3f1-927b-44b1-b63b-3f6503b4fee3-kube-api-access-qb8fk\") pod \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") "
Nov 02 02:06:06.061667 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.061652    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-data\") pod \"444174f7-6dd7-411d-b625-e3edd0f214ce\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") "
Nov 02 02:06:06.061818 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.061685    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-workload-certs\") pod \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") "
Nov 02 02:06:06.061818 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.061714    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-podinfo\") pod \"444174f7-6dd7-411d-b625-e3edd0f214ce\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") "
Nov 02 02:06:06.061818 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.061773    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-envoy\") pod \"444174f7-6dd7-411d-b625-e3edd0f214ce\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") "
Nov 02 02:06:06.061984 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.061859    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-envoy\") pod \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") "
Nov 02 02:06:06.061984 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.061908    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-credential-socket\") pod \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") "
Nov 02 02:06:06.061984 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.061941    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-workload-certs\") pod \"444174f7-6dd7-411d-b625-e3edd0f214ce\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") "
Nov 02 02:06:06.061984 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.061949    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-data" pod "90baf3f1-927b-44b1-b63b-3f6503b4fee3" (UID: "90baf3f1-927b-44b1-b63b-3f6503b4fee3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:06.061984 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.061957    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-data" pod "444174f7-6dd7-411d-b625-e3edd0f214ce" (UID: "444174f7-6dd7-411d-b625-e3edd0f214ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:06.062257 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.061986    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/444174f7-6dd7-411d-b625-e3edd0f214ce-kube-api-access-qjrgc\" (UniqueName: \"kubernetes.io/projected/444174f7-6dd7-411d-b625-e3edd0f214ce-kube-api-access-qjrgc\") pod \"444174f7-6dd7-411d-b625-e3edd0f214ce\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") "
Nov 02 02:06:06.062257 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.062110    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-podinfo\") pod \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") "
Nov 02 02:06:06.062257 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.062191    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istiod-ca-cert\") pod \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\" (UID: \"90baf3f1-927b-44b1-b63b-3f6503b4fee3\") "
Nov 02 02:06:06.062257 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.062245    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-workload-socket\") pod \"444174f7-6dd7-411d-b625-e3edd0f214ce\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") "
Nov 02 02:06:06.062456 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.062278    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/444174f7-6dd7-411d-b625-e3edd0f214ce-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/444174f7-6dd7-411d-b625-e3edd0f214ce-istiod-ca-cert\") pod \"444174f7-6dd7-411d-b625-e3edd0f214ce\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") "
Nov 02 02:06:06.062456 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.062278    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-credential-socket" pod "90baf3f1-927b-44b1-b63b-3f6503b4fee3" (UID: "90baf3f1-927b-44b1-b63b-3f6503b4fee3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:06.062456 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.062309    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-credential-socket\") pod \"444174f7-6dd7-411d-b625-e3edd0f214ce\" (UID: \"444174f7-6dd7-411d-b625-e3edd0f214ce\") "
Nov 02 02:06:06.062611 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.062503    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-workload-certs" pod "90baf3f1-927b-44b1-b63b-3f6503b4fee3" (UID: "90baf3f1-927b-44b1-b63b-3f6503b4fee3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:06.062932 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.062716    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-credential-socket" pod "444174f7-6dd7-411d-b625-e3edd0f214ce" (UID: "444174f7-6dd7-411d-b625-e3edd0f214ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:06.062932 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.062903    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-workload-certs" pod "444174f7-6dd7-411d-b625-e3edd0f214ce" (UID: "444174f7-6dd7-411d-b625-e3edd0f214ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:06.063616 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.063377    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-workload-socket" pod "444174f7-6dd7-411d-b625-e3edd0f214ce" (UID: "444174f7-6dd7-411d-b625-e3edd0f214ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:06.063616 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.063514    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/444174f7-6dd7-411d-b625-e3edd0f214ce-istiod-ca-cert" pod "444174f7-6dd7-411d-b625-e3edd0f214ce" (UID: "444174f7-6dd7-411d-b625-e3edd0f214ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:06:06.063775 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.063688    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-workload-socket" pod "90baf3f1-927b-44b1-b63b-3f6503b4fee3" (UID: "90baf3f1-927b-44b1-b63b-3f6503b4fee3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:06.065185 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.064320    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.065185 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.064561    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-token" pod "90baf3f1-927b-44b1-b63b-3f6503b4fee3" (UID: "90baf3f1-927b-44b1-b63b-3f6503b4fee3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:06:06.065185 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.064900    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istiod-ca-cert" pod "90baf3f1-927b-44b1-b63b-3f6503b4fee3" (UID: "90baf3f1-927b-44b1-b63b-3f6503b4fee3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:06:06.065185 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.065085    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.065185 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.065106    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.065185 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.065139    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.065185 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.065154    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.065185 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.065169    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.066524 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.065584    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90baf3f1-927b-44b1-b63b-3f6503b4fee3-kube-api-access-qb8fk" pod "90baf3f1-927b-44b1-b63b-3f6503b4fee3" (UID: "90baf3f1-927b-44b1-b63b-3f6503b4fee3"). InnerVolumeSpecName "kube-api-access-qb8fk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:06:06.067483 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.067456    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-envoy" pod "444174f7-6dd7-411d-b625-e3edd0f214ce" (UID: "444174f7-6dd7-411d-b625-e3edd0f214ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:06.067942 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.067908    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-podinfo" pod "444174f7-6dd7-411d-b625-e3edd0f214ce" (UID: "444174f7-6dd7-411d-b625-e3edd0f214ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:06:06.068043 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.067993    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-podinfo" pod "90baf3f1-927b-44b1-b63b-3f6503b4fee3" (UID: "90baf3f1-927b-44b1-b63b-3f6503b4fee3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:06:06.068255 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.068230    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-token" pod "444174f7-6dd7-411d-b625-e3edd0f214ce" (UID: "444174f7-6dd7-411d-b625-e3edd0f214ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:06:06.069487 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.069459    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-envoy" pod "90baf3f1-927b-44b1-b63b-3f6503b4fee3" (UID: "90baf3f1-927b-44b1-b63b-3f6503b4fee3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:06.069589 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.069558    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/444174f7-6dd7-411d-b625-e3edd0f214ce-kube-api-access-qjrgc" pod "444174f7-6dd7-411d-b625-e3edd0f214ce" (UID: "444174f7-6dd7-411d-b625-e3edd0f214ce"). InnerVolumeSpecName "kube-api-access-qjrgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:06:06.168381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.168143    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjrgc\" (UniqueName: \"kubernetes.io/projected/444174f7-6dd7-411d-b625-e3edd0f214ce-kube-api-access-qjrgc\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.168381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.168183    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.168381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.168198    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.168381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.168211    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.168381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.168224    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/444174f7-6dd7-411d-b625-e3edd0f214ce-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.168381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.168236    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.168381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.168250    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.168381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.168264    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.168381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.168277    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qb8fk\" (UniqueName: \"kubernetes.io/projected/90baf3f1-927b-44b1-b63b-3f6503b4fee3-kube-api-access-qb8fk\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.168381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.168292    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.168381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.168306    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/444174f7-6dd7-411d-b625-e3edd0f214ce-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.168381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.168320    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90baf3f1-927b-44b1-b63b-3f6503b4fee3-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:06.260280 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.260246    4515 generic.go:356] "Generic (PLEG): container finished" podID="90baf3f1-927b-44b1-b63b-3f6503b4fee3" containerID="dd411abacd787178ba930ac6ef5a43f602dd22456d7b8587607e7889d0f8f4cb" exitCode=0
Nov 02 02:06:06.260722 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.260294    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"
Nov 02 02:06:06.260722 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.260329    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4" event={"ID":"90baf3f1-927b-44b1-b63b-3f6503b4fee3","Type":"ContainerDied","Data":"dd411abacd787178ba930ac6ef5a43f602dd22456d7b8587607e7889d0f8f4cb"}
Nov 02 02:06:06.260722 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.260381    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4" event={"ID":"90baf3f1-927b-44b1-b63b-3f6503b4fee3","Type":"ContainerDied","Data":"e4e4024ec7065840f92b0a1525b82515527626acf26383c7060522328724b33a"}
Nov 02 02:06:06.260722 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.260411    4515 scope.go:122] "RemoveContainer" containerID="dd411abacd787178ba930ac6ef5a43f602dd22456d7b8587607e7889d0f8f4cb"
Nov 02 02:06:06.261758 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.261736    4515 generic.go:356] "Generic (PLEG): container finished" podID="444174f7-6dd7-411d-b625-e3edd0f214ce" containerID="67d599c809cf9b757f0ab84bf60b9d4172ba11a6071d9fddae72a551bfad3c18" exitCode=0
Nov 02 02:06:06.261837 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.261790    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"
Nov 02 02:06:06.261837 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.261820    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx" event={"ID":"444174f7-6dd7-411d-b625-e3edd0f214ce","Type":"ContainerDied","Data":"67d599c809cf9b757f0ab84bf60b9d4172ba11a6071d9fddae72a551bfad3c18"}
Nov 02 02:06:06.261921 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.261875    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx" event={"ID":"444174f7-6dd7-411d-b625-e3edd0f214ce","Type":"ContainerDied","Data":"28c394a1bafeb5aacf173f2ed7d7d6a3ccc9b47cadaf17d3d110896cf78baeda"}
Nov 02 02:06:06.278141 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.278105    4515 scope.go:122] "RemoveContainer" containerID="dd411abacd787178ba930ac6ef5a43f602dd22456d7b8587607e7889d0f8f4cb"
Nov 02 02:06:06.278536 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:06:06.278513    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd411abacd787178ba930ac6ef5a43f602dd22456d7b8587607e7889d0f8f4cb\": container with ID starting with dd411abacd787178ba930ac6ef5a43f602dd22456d7b8587607e7889d0f8f4cb not found: ID does not exist" containerID="dd411abacd787178ba930ac6ef5a43f602dd22456d7b8587607e7889d0f8f4cb"
Nov 02 02:06:06.278606 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.278550    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd411abacd787178ba930ac6ef5a43f602dd22456d7b8587607e7889d0f8f4cb"} err="failed to get container status \"dd411abacd787178ba930ac6ef5a43f602dd22456d7b8587607e7889d0f8f4cb\": rpc error: code = NotFound desc = could not find container \"dd411abacd787178ba930ac6ef5a43f602dd22456d7b8587607e7889d0f8f4cb\": container with ID starting with dd411abacd787178ba930ac6ef5a43f602dd22456d7b8587607e7889d0f8f4cb not found: ID does not exist"
Nov 02 02:06:06.278606 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.278571    4515 scope.go:122] "RemoveContainer" containerID="67d599c809cf9b757f0ab84bf60b9d4172ba11a6071d9fddae72a551bfad3c18"
Nov 02 02:06:06.289974 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.287989    4515 scope.go:122] "RemoveContainer" containerID="67d599c809cf9b757f0ab84bf60b9d4172ba11a6071d9fddae72a551bfad3c18"
Nov 02 02:06:06.290433 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:06:06.290401    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67d599c809cf9b757f0ab84bf60b9d4172ba11a6071d9fddae72a551bfad3c18\": container with ID starting with 67d599c809cf9b757f0ab84bf60b9d4172ba11a6071d9fddae72a551bfad3c18 not found: ID does not exist" containerID="67d599c809cf9b757f0ab84bf60b9d4172ba11a6071d9fddae72a551bfad3c18"
Nov 02 02:06:06.290517 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.290441    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67d599c809cf9b757f0ab84bf60b9d4172ba11a6071d9fddae72a551bfad3c18"} err="failed to get container status \"67d599c809cf9b757f0ab84bf60b9d4172ba11a6071d9fddae72a551bfad3c18\": rpc error: code = NotFound desc = could not find container \"67d599c809cf9b757f0ab84bf60b9d4172ba11a6071d9fddae72a551bfad3c18\": container with ID starting with 67d599c809cf9b757f0ab84bf60b9d4172ba11a6071d9fddae72a551bfad3c18 not found: ID does not exist"
Nov 02 02:06:06.425442 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.425362    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"]
Nov 02 02:06:06.428799 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.428777    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-c9mwx"]
Nov 02 02:06:06.538399 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.538370    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"]
Nov 02 02:06:06.542961 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.542933    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-jlgt4"]
Nov 02 02:06:06.952931 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.952899    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a2f9447-a086-4541-8a70-4e6dc5b98ac0" path="/var/lib/kubelet/pods/3a2f9447-a086-4541-8a70-4e6dc5b98ac0/volumes"
Nov 02 02:06:06.953408 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.953393    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3f4a20c6-793d-44fd-988e-a1b950a71ea7" path="/var/lib/kubelet/pods/3f4a20c6-793d-44fd-988e-a1b950a71ea7/volumes"
Nov 02 02:06:06.953832 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.953819    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="444174f7-6dd7-411d-b625-e3edd0f214ce" path="/var/lib/kubelet/pods/444174f7-6dd7-411d-b625-e3edd0f214ce/volumes"
Nov 02 02:06:06.954557 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:06.954541    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90baf3f1-927b-44b1-b63b-3f6503b4fee3" path="/var/lib/kubelet/pods/90baf3f1-927b-44b1-b63b-3f6503b4fee3/volumes"
Nov 02 02:06:09.966594 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:06:09.966547    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:06:10.012952 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.012894    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 02:06:10.023780 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.023753    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istiod-ca-cert\") pod \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") "
Nov 02 02:06:10.023919 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.023807    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-envoy\") pod \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") "
Nov 02 02:06:10.023919 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.023878    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-token\" (UniqueName: \"kubernetes.io/projected/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-token\") pod \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") "
Nov 02 02:06:10.023919 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.023909    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-workload-socket\") pod \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") "
Nov 02 02:06:10.024086 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.023982    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-credential-socket\") pod \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") "
Nov 02 02:06:10.024086 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.024067    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istiod-ca-cert" pod "dbce4552-66b5-4a8f-ad14-edacdc87ea52" (UID: "dbce4552-66b5-4a8f-ad14-edacdc87ea52"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:06:10.024205 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.024130    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-podinfo\") pod \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") "
Nov 02 02:06:10.024205 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.024175    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dbce4552-66b5-4a8f-ad14-edacdc87ea52-kube-api-access-nnffw\" (UniqueName: \"kubernetes.io/projected/dbce4552-66b5-4a8f-ad14-edacdc87ea52-kube-api-access-nnffw\") pod \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") "
Nov 02 02:06:10.024317 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.024210    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-workload-certs\") pod \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") "
Nov 02 02:06:10.024317 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.024274    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-workload-socket" pod "dbce4552-66b5-4a8f-ad14-edacdc87ea52" (UID: "dbce4552-66b5-4a8f-ad14-edacdc87ea52"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:10.024401 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.024302    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-credential-socket" pod "dbce4552-66b5-4a8f-ad14-edacdc87ea52" (UID: "dbce4552-66b5-4a8f-ad14-edacdc87ea52"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:10.024401 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.024355    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-data\") pod \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\" (UID: \"dbce4552-66b5-4a8f-ad14-edacdc87ea52\") "
Nov 02 02:06:10.024582 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.024558    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-workload-certs" pod "dbce4552-66b5-4a8f-ad14-edacdc87ea52" (UID: "dbce4552-66b5-4a8f-ad14-edacdc87ea52"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:10.024692 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.024671    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-data" pod "dbce4552-66b5-4a8f-ad14-edacdc87ea52" (UID: "dbce4552-66b5-4a8f-ad14-edacdc87ea52"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:10.025277 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.025214    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:10.025277 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.025248    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:10.025277 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.025265    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:10.025277 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.025278    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:10.025457 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.025293    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:10.026269 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.026247    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-envoy" pod "dbce4552-66b5-4a8f-ad14-edacdc87ea52" (UID: "dbce4552-66b5-4a8f-ad14-edacdc87ea52"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:10.026269 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.026258    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-token" pod "dbce4552-66b5-4a8f-ad14-edacdc87ea52" (UID: "dbce4552-66b5-4a8f-ad14-edacdc87ea52"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:06:10.026666 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.026652    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-podinfo" pod "dbce4552-66b5-4a8f-ad14-edacdc87ea52" (UID: "dbce4552-66b5-4a8f-ad14-edacdc87ea52"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:06:10.026666 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.026658    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbce4552-66b5-4a8f-ad14-edacdc87ea52-kube-api-access-nnffw" pod "dbce4552-66b5-4a8f-ad14-edacdc87ea52" (UID: "dbce4552-66b5-4a8f-ad14-edacdc87ea52"). InnerVolumeSpecName "kube-api-access-nnffw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:06:10.126107 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.126073    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:10.126107 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.126100    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:10.126107 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.126110    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbce4552-66b5-4a8f-ad14-edacdc87ea52-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:10.126385 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.126151    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnffw\" (UniqueName: \"kubernetes.io/projected/dbce4552-66b5-4a8f-ad14-edacdc87ea52-kube-api-access-nnffw\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:10.287874 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.287787    4515 generic.go:356] "Generic (PLEG): container finished" podID="dbce4552-66b5-4a8f-ad14-edacdc87ea52" containerID="a78258ff2753e157bc6fcf46d270506c66560411ef315e5ecb5e6d335aaf5ab4" exitCode=0
Nov 02 02:06:10.287874 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.287841    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"
Nov 02 02:06:10.288066 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.287869    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg" event={"ID":"dbce4552-66b5-4a8f-ad14-edacdc87ea52","Type":"ContainerDied","Data":"a78258ff2753e157bc6fcf46d270506c66560411ef315e5ecb5e6d335aaf5ab4"}
Nov 02 02:06:10.288066 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.287932    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg" event={"ID":"dbce4552-66b5-4a8f-ad14-edacdc87ea52","Type":"ContainerDied","Data":"0a19c313d942d1d73a3edd1fd848a0e588cc385a7048091873a64a2b93c87bb9"}
Nov 02 02:06:10.288066 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.287956    4515 scope.go:122] "RemoveContainer" containerID="a78258ff2753e157bc6fcf46d270506c66560411ef315e5ecb5e6d335aaf5ab4"
Nov 02 02:06:10.302508 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.302481    4515 scope.go:122] "RemoveContainer" containerID="a78258ff2753e157bc6fcf46d270506c66560411ef315e5ecb5e6d335aaf5ab4"
Nov 02 02:06:10.302832 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:06:10.302813    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a78258ff2753e157bc6fcf46d270506c66560411ef315e5ecb5e6d335aaf5ab4\": container with ID starting with a78258ff2753e157bc6fcf46d270506c66560411ef315e5ecb5e6d335aaf5ab4 not found: ID does not exist" containerID="a78258ff2753e157bc6fcf46d270506c66560411ef315e5ecb5e6d335aaf5ab4"
Nov 02 02:06:10.302900 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.302843    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a78258ff2753e157bc6fcf46d270506c66560411ef315e5ecb5e6d335aaf5ab4"} err="failed to get container status \"a78258ff2753e157bc6fcf46d270506c66560411ef315e5ecb5e6d335aaf5ab4\": rpc error: code = NotFound desc = could not find container \"a78258ff2753e157bc6fcf46d270506c66560411ef315e5ecb5e6d335aaf5ab4\": container with ID starting with a78258ff2753e157bc6fcf46d270506c66560411ef315e5ecb5e6d335aaf5ab4 not found: ID does not exist"
Nov 02 02:06:10.452155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.452105    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"]
Nov 02 02:06:10.456808 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.456781    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xs4xg"]
Nov 02 02:06:10.954052 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:10.954022    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dbce4552-66b5-4a8f-ad14-edacdc87ea52" path="/var/lib/kubelet/pods/dbce4552-66b5-4a8f-ad14-edacdc87ea52/volumes"
Nov 02 02:06:15.903339 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:15.903305    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"]
Nov 02 02:06:15.903739 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:15.903530    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx" podUID="98b67a95-2731-4fd4-83f5-f8a161e03a19" containerName="istio-proxy" containerID="cri-o://7d22f257007f1dfd99c176c94d7c2b4f18cc7de73db0def2afe5e57382bc2c1e" gracePeriod=30
Nov 02 02:06:16.142320 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:16.142287    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"]
Nov 02 02:06:16.142543 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:16.142521    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6" podUID="1090bbf7-6cf8-43d7-b777-5bcaa301f76d" containerName="istio-proxy" containerID="cri-o://ec0f0a062766d2d608712fe1ee6612b18e4b10670cd118c78ddbe0f97f4f17ef" gracePeriod=30
Nov 02 02:06:16.299068 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:16.299030    4515 patch_prober.go:29] interesting pod/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 02:06:16.299279 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:16.299090    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6" podUID="1090bbf7-6cf8-43d7-b777-5bcaa301f76d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 02:06:18.808876 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:18.808473    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 02:06:18.808876 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:18.808539    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx" podUID="98b67a95-2731-4fd4-83f5-f8a161e03a19" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 02:06:21.238790 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.238758    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 02:06:21.352626 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.352588    4515 generic.go:356] "Generic (PLEG): container finished" podID="1090bbf7-6cf8-43d7-b777-5bcaa301f76d" containerID="ec0f0a062766d2d608712fe1ee6612b18e4b10670cd118c78ddbe0f97f4f17ef" exitCode=0
Nov 02 02:06:21.352767 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.352653    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6" event={"ID":"1090bbf7-6cf8-43d7-b777-5bcaa301f76d","Type":"ContainerDied","Data":"ec0f0a062766d2d608712fe1ee6612b18e4b10670cd118c78ddbe0f97f4f17ef"}
Nov 02 02:06:21.354173 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.354140    4515 generic.go:356] "Generic (PLEG): container finished" podID="98b67a95-2731-4fd4-83f5-f8a161e03a19" containerID="7d22f257007f1dfd99c176c94d7c2b4f18cc7de73db0def2afe5e57382bc2c1e" exitCode=0
Nov 02 02:06:21.354311 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.354199    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx" event={"ID":"98b67a95-2731-4fd4-83f5-f8a161e03a19","Type":"ContainerDied","Data":"7d22f257007f1dfd99c176c94d7c2b4f18cc7de73db0def2afe5e57382bc2c1e"}
Nov 02 02:06:21.354311 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.354209    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"
Nov 02 02:06:21.354311 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.354240    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx" event={"ID":"98b67a95-2731-4fd4-83f5-f8a161e03a19","Type":"ContainerDied","Data":"0b0bed73ae93bc00fe1c8b6bd290e53692ac5c5544bc9ce8882d2ca81c71c70e"}
Nov 02 02:06:21.354311 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.354268    4515 scope.go:122] "RemoveContainer" containerID="7d22f257007f1dfd99c176c94d7c2b4f18cc7de73db0def2afe5e57382bc2c1e"
Nov 02 02:06:21.362752 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.362725    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/98b67a95-2731-4fd4-83f5-f8a161e03a19-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98b67a95-2731-4fd4-83f5-f8a161e03a19-istiod-ca-cert\") pod \"98b67a95-2731-4fd4-83f5-f8a161e03a19\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") "
Nov 02 02:06:21.362885 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.362855    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-workload-socket\") pod \"98b67a95-2731-4fd4-83f5-f8a161e03a19\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") "
Nov 02 02:06:21.362950 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.362889    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-workload-certs\") pod \"98b67a95-2731-4fd4-83f5-f8a161e03a19\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") "
Nov 02 02:06:21.362950 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.362929    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98b67a95-2731-4fd4-83f5-f8a161e03a19-kube-api-access-jxxgq\" (UniqueName: \"kubernetes.io/projected/98b67a95-2731-4fd4-83f5-f8a161e03a19-kube-api-access-jxxgq\") pod \"98b67a95-2731-4fd4-83f5-f8a161e03a19\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") "
Nov 02 02:06:21.363089 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.363071    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-podinfo\") pod \"98b67a95-2731-4fd4-83f5-f8a161e03a19\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") "
Nov 02 02:06:21.363167 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.363150    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-envoy\") pod \"98b67a95-2731-4fd4-83f5-f8a161e03a19\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") "
Nov 02 02:06:21.363248 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.363184    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-credential-socket\") pod \"98b67a95-2731-4fd4-83f5-f8a161e03a19\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") "
Nov 02 02:06:21.363325 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.363239    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-workload-socket" pod "98b67a95-2731-4fd4-83f5-f8a161e03a19" (UID: "98b67a95-2731-4fd4-83f5-f8a161e03a19"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:21.363325 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.363267    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-token\" (UniqueName: \"kubernetes.io/projected/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-token\") pod \"98b67a95-2731-4fd4-83f5-f8a161e03a19\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") "
Nov 02 02:06:21.363325 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.363319    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-data\") pod \"98b67a95-2731-4fd4-83f5-f8a161e03a19\" (UID: \"98b67a95-2731-4fd4-83f5-f8a161e03a19\") "
Nov 02 02:06:21.363533 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.363468    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-credential-socket" pod "98b67a95-2731-4fd4-83f5-f8a161e03a19" (UID: "98b67a95-2731-4fd4-83f5-f8a161e03a19"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:21.363640 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.363621    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-workload-certs" pod "98b67a95-2731-4fd4-83f5-f8a161e03a19" (UID: "98b67a95-2731-4fd4-83f5-f8a161e03a19"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:21.363779 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.363753    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-data" pod "98b67a95-2731-4fd4-83f5-f8a161e03a19" (UID: "98b67a95-2731-4fd4-83f5-f8a161e03a19"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:21.363891 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.363868    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98b67a95-2731-4fd4-83f5-f8a161e03a19-istiod-ca-cert" pod "98b67a95-2731-4fd4-83f5-f8a161e03a19" (UID: "98b67a95-2731-4fd4-83f5-f8a161e03a19"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:06:21.364063 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.363981    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.364063 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.363999    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.364063 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.364012    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98b67a95-2731-4fd4-83f5-f8a161e03a19-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.364063 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.364026    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.364063 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.364040    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.364915 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.364895    4515 scope.go:122] "RemoveContainer" containerID="7d22f257007f1dfd99c176c94d7c2b4f18cc7de73db0def2afe5e57382bc2c1e"
Nov 02 02:06:21.365307 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:06:21.365277    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d22f257007f1dfd99c176c94d7c2b4f18cc7de73db0def2afe5e57382bc2c1e\": container with ID starting with 7d22f257007f1dfd99c176c94d7c2b4f18cc7de73db0def2afe5e57382bc2c1e not found: ID does not exist" containerID="7d22f257007f1dfd99c176c94d7c2b4f18cc7de73db0def2afe5e57382bc2c1e"
Nov 02 02:06:21.365383 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.365324    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d22f257007f1dfd99c176c94d7c2b4f18cc7de73db0def2afe5e57382bc2c1e"} err="failed to get container status \"7d22f257007f1dfd99c176c94d7c2b4f18cc7de73db0def2afe5e57382bc2c1e\": rpc error: code = NotFound desc = could not find container \"7d22f257007f1dfd99c176c94d7c2b4f18cc7de73db0def2afe5e57382bc2c1e\": container with ID starting with 7d22f257007f1dfd99c176c94d7c2b4f18cc7de73db0def2afe5e57382bc2c1e not found: ID does not exist"
Nov 02 02:06:21.366049 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.366026    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-podinfo" pod "98b67a95-2731-4fd4-83f5-f8a161e03a19" (UID: "98b67a95-2731-4fd4-83f5-f8a161e03a19"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:06:21.367019 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.366991    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-token" pod "98b67a95-2731-4fd4-83f5-f8a161e03a19" (UID: "98b67a95-2731-4fd4-83f5-f8a161e03a19"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:06:21.367090 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.367054    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-envoy" pod "98b67a95-2731-4fd4-83f5-f8a161e03a19" (UID: "98b67a95-2731-4fd4-83f5-f8a161e03a19"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:21.367171 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.367092    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98b67a95-2731-4fd4-83f5-f8a161e03a19-kube-api-access-jxxgq" pod "98b67a95-2731-4fd4-83f5-f8a161e03a19" (UID: "98b67a95-2731-4fd4-83f5-f8a161e03a19"). InnerVolumeSpecName "kube-api-access-jxxgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:06:21.393639 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.393619    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 02:06:21.465482 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.465447    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-podinfo\") pod \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") "
Nov 02 02:06:21.465669 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.465644    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istiod-ca-cert\") pod \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") "
Nov 02 02:06:21.465824 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.465752    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-workload-socket\") pod \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") "
Nov 02 02:06:21.465824 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.465788    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-data\") pod \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") "
Nov 02 02:06:21.465949 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.465822    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-token\" (UniqueName: \"kubernetes.io/projected/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-token\") pod \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") "
Nov 02 02:06:21.465949 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.465870    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-credential-socket\") pod \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") "
Nov 02 02:06:21.465949 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.465918    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-workload-certs\") pod \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") "
Nov 02 02:06:21.466102 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.465958    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-envoy\") pod \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") "
Nov 02 02:06:21.466102 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.466009    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-kube-api-access-4fdgr\" (UniqueName: \"kubernetes.io/projected/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-kube-api-access-4fdgr\") pod \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\" (UID: \"1090bbf7-6cf8-43d7-b777-5bcaa301f76d\") "
Nov 02 02:06:21.466231 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.466187    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istiod-ca-cert" pod "1090bbf7-6cf8-43d7-b777-5bcaa301f76d" (UID: "1090bbf7-6cf8-43d7-b777-5bcaa301f76d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:06:21.466469 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.466446    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-credential-socket" pod "1090bbf7-6cf8-43d7-b777-5bcaa301f76d" (UID: "1090bbf7-6cf8-43d7-b777-5bcaa301f76d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:21.466606 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.466585    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.466675 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.466616    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.466675 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.466636    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxxgq\" (UniqueName: \"kubernetes.io/projected/98b67a95-2731-4fd4-83f5-f8a161e03a19-kube-api-access-jxxgq\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.466675 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.466650    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.466675 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.466665    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.466888 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.466680    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98b67a95-2731-4fd4-83f5-f8a161e03a19-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.466888 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.466615    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-workload-socket" pod "1090bbf7-6cf8-43d7-b777-5bcaa301f76d" (UID: "1090bbf7-6cf8-43d7-b777-5bcaa301f76d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:21.466998 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.466951    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-workload-certs" pod "1090bbf7-6cf8-43d7-b777-5bcaa301f76d" (UID: "1090bbf7-6cf8-43d7-b777-5bcaa301f76d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:21.467048 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.467022    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-data" pod "1090bbf7-6cf8-43d7-b777-5bcaa301f76d" (UID: "1090bbf7-6cf8-43d7-b777-5bcaa301f76d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:21.468213 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.468186    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-podinfo" pod "1090bbf7-6cf8-43d7-b777-5bcaa301f76d" (UID: "1090bbf7-6cf8-43d7-b777-5bcaa301f76d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:06:21.468378 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.468355    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-envoy" pod "1090bbf7-6cf8-43d7-b777-5bcaa301f76d" (UID: "1090bbf7-6cf8-43d7-b777-5bcaa301f76d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:06:21.468554 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.468537    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-token" pod "1090bbf7-6cf8-43d7-b777-5bcaa301f76d" (UID: "1090bbf7-6cf8-43d7-b777-5bcaa301f76d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:06:21.468997 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.468979    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-kube-api-access-4fdgr" pod "1090bbf7-6cf8-43d7-b777-5bcaa301f76d" (UID: "1090bbf7-6cf8-43d7-b777-5bcaa301f76d"). InnerVolumeSpecName "kube-api-access-4fdgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:06:21.574079 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.573184    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.574079 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.573224    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.574079 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.573245    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.574079 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.573266    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.574079 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.573280    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.574079 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.573295    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.574079 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.573309    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fdgr\" (UniqueName: \"kubernetes.io/projected/1090bbf7-6cf8-43d7-b777-5bcaa301f76d-kube-api-access-4fdgr\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:06:21.775636 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.775548    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"]
Nov 02 02:06:21.779542 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:21.779508    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-s4ngx"]
Nov 02 02:06:22.361942 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:22.361917    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"
Nov 02 02:06:22.361942 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:22.361934    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6" event={"ID":"1090bbf7-6cf8-43d7-b777-5bcaa301f76d","Type":"ContainerDied","Data":"bd94eeb983dbd783544e1457030777930d6c3106e7bf0d48b6a6759958150b01"}
Nov 02 02:06:22.362462 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:22.361992    4515 scope.go:122] "RemoveContainer" containerID="ec0f0a062766d2d608712fe1ee6612b18e4b10670cd118c78ddbe0f97f4f17ef"
Nov 02 02:06:22.528465 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:22.528434    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"]
Nov 02 02:06:22.577697 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:22.577667    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-dbe849-openshift-default-579655d747-qbcd6"]
Nov 02 02:06:22.954025 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:22.953988    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1090bbf7-6cf8-43d7-b777-5bcaa301f76d" path="/var/lib/kubelet/pods/1090bbf7-6cf8-43d7-b777-5bcaa301f76d/volumes"
Nov 02 02:06:22.954499 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:22.954485    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="98b67a95-2731-4fd4-83f5-f8a161e03a19" path="/var/lib/kubelet/pods/98b67a95-2731-4fd4-83f5-f8a161e03a19/volumes"
Nov 02 02:06:39.967695 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:06:39.967653    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:06:51.177863 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.177822    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 02:06:51.182686 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.182668    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 02:06:51.694154 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694101    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"]
Nov 02 02:06:51.694594 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694578    4515 cpu_manager.go:414] "containerMap: removing container" podUID="4d8a0143-8900-44c1-aa7c-bf3b8503a938" containerName="test-curl-7028-1f4df2"
Nov 02 02:06:51.694642 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694595    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d8a0143-8900-44c1-aa7c-bf3b8503a938" containerName="test-curl-7028-1f4df2"
Nov 02 02:06:51.694642 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694609    4515 cpu_manager.go:414] "containerMap: removing container" podUID="47069c5d-070f-425b-ba94-a69d0585fd29" containerName="istio-proxy"
Nov 02 02:06:51.694642 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694615    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47069c5d-070f-425b-ba94-a69d0585fd29" containerName="istio-proxy"
Nov 02 02:06:51.694642 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694624    4515 cpu_manager.go:414] "containerMap: removing container" podUID="3a2f9447-a086-4541-8a70-4e6dc5b98ac0" containerName="istio-proxy"
Nov 02 02:06:51.694642 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694629    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a2f9447-a086-4541-8a70-4e6dc5b98ac0" containerName="istio-proxy"
Nov 02 02:06:51.694642 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694643    4515 cpu_manager.go:414] "containerMap: removing container" podUID="444174f7-6dd7-411d-b625-e3edd0f214ce" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694649    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="444174f7-6dd7-411d-b625-e3edd0f214ce" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694656    4515 cpu_manager.go:414] "containerMap: removing container" podUID="90baf3f1-927b-44b1-b63b-3f6503b4fee3" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694661    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90baf3f1-927b-44b1-b63b-3f6503b4fee3" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694671    4515 cpu_manager.go:414] "containerMap: removing container" podUID="1090bbf7-6cf8-43d7-b777-5bcaa301f76d" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694676    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1090bbf7-6cf8-43d7-b777-5bcaa301f76d" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694685    4515 cpu_manager.go:414] "containerMap: removing container" podUID="98b67a95-2731-4fd4-83f5-f8a161e03a19" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694689    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="98b67a95-2731-4fd4-83f5-f8a161e03a19" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694697    4515 cpu_manager.go:414] "containerMap: removing container" podUID="3f4a20c6-793d-44fd-988e-a1b950a71ea7" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694701    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f4a20c6-793d-44fd-988e-a1b950a71ea7" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694710    4515 cpu_manager.go:414] "containerMap: removing container" podUID="dbce4552-66b5-4a8f-ad14-edacdc87ea52" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694714    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbce4552-66b5-4a8f-ad14-edacdc87ea52" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694783    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d8a0143-8900-44c1-aa7c-bf3b8503a938" containerName="test-curl-7028-1f4df2"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694792    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="47069c5d-070f-425b-ba94-a69d0585fd29" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694799    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a2f9447-a086-4541-8a70-4e6dc5b98ac0" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694807    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="444174f7-6dd7-411d-b625-e3edd0f214ce" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694811    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="90baf3f1-927b-44b1-b63b-3f6503b4fee3" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694817    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="1090bbf7-6cf8-43d7-b777-5bcaa301f76d" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694823    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="98b67a95-2731-4fd4-83f5-f8a161e03a19" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694829    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f4a20c6-793d-44fd-988e-a1b950a71ea7" containerName="istio-proxy"
Nov 02 02:06:51.694856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.694834    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="dbce4552-66b5-4a8f-ad14-edacdc87ea52" containerName="istio-proxy"
Nov 02 02:06:51.699354 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.699334    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.835182 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.814309    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-1c942d-openshift-default-dockercfg-kf7x7\""
Nov 02 02:06:51.835182 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.829481    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-podinfo\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.835182 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.829667    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwj45\" (UniqueName: \"kubernetes.io/projected/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-kube-api-access-lwj45\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.835182 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.829885    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-credential-socket\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.835182 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.829944    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-workload-socket\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.835182 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.830226    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-data\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.835182 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.830270    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-workload-certs\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.835182 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.830343    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-token\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.835182 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.830451    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-envoy\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.835182 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.830570    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istiod-ca-cert\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.873142 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.870834    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"]
Nov 02 02:06:51.941352 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.937094    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-data\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.941352 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.940530    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-workload-certs\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.941352 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.940570    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-token\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.941352 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.940612    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-envoy\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.941352 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.940689    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istiod-ca-cert\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.941352 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.940741    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-podinfo\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.941352 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.940797    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwj45\" (UniqueName: \"kubernetes.io/projected/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-kube-api-access-lwj45\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.941352 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.940826    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-credential-socket\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.941352 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.940865    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-workload-socket\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.941352 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.941353    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-workload-socket\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.942191 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.937827    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-data\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.942191 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.941610    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-workload-certs\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.943332 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.942804    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istiod-ca-cert\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.943332 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.943295    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-credential-socket\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.963652 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.953729    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-podinfo\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:51.979426 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:51.979395    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-envoy\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:52.006338 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:52.004806    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwj45\" (UniqueName: \"kubernetes.io/projected/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-kube-api-access-lwj45\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:52.006338 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:52.006106    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-token\") pod \"e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") " pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:52.018564 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:52.018535    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:52.459965 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:52.459923    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"]
Nov 02 02:06:52.476159 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:52.476095    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:06:52.476288 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:52.476206    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:06:52.476288 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:52.476247    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:06:52.558212 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:52.558171    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh" event={"ID":"7bcde5e3-f9bf-4724-8a21-c6a63c13df32","Type":"ContainerStarted","Data":"e09a933d76d232242df3a76cfe337298e2ca7c18e047811e392cdec8f3f56853"}
Nov 02 02:06:53.565807 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:53.565776    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh" event={"ID":"7bcde5e3-f9bf-4724-8a21-c6a63c13df32","Type":"ContainerStarted","Data":"8a17a1bc19f12cee6a79978f19371b61dfa0d865648117fe4c5f2390ddff86e5"}
Nov 02 02:06:53.764392 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:53.764340    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh" podStartSLOduration=2.764325491 podStartE2EDuration="2.764325491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:06:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:06:53.735256733 +0000 UTC m=+2102.947626543" watchObservedRunningTime="2026-11-02 02:06:53.764325491 +0000 UTC m=+2102.976695361"
Nov 02 02:06:54.019575 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:54.019541    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:54.025373 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:54.025348    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:54.571263 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:54.571232    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:06:54.572210 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:06:54.572187    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:07:09.960423 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:07:09.960340    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:07:29.576822 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:29.576790    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"]
Nov 02 02:07:29.577402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:29.577011    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm" podUID="fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" containerName="istio-proxy" containerID="cri-o://23090ea8c8193ff14c7651e63cddc3d9ce2bae01a5790b0263741c74e76a985d" gracePeriod=30
Nov 02 02:07:30.209764 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:30.209723    4515 patch_prober.go:29] interesting pod/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 02:07:30.209961 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:30.209789    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm" podUID="fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 02:07:34.824385 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.824357    4515 generic.go:356] "Generic (PLEG): container finished" podID="fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" containerID="23090ea8c8193ff14c7651e63cddc3d9ce2bae01a5790b0263741c74e76a985d" exitCode=0
Nov 02 02:07:34.824704 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.824413    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm" event={"ID":"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004","Type":"ContainerDied","Data":"23090ea8c8193ff14c7651e63cddc3d9ce2bae01a5790b0263741c74e76a985d"}
Nov 02 02:07:34.824704 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.824473    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm" event={"ID":"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004","Type":"ContainerDied","Data":"c18b0af6b09a2191d5ef9b3fa3a276d8607c299f83a45d1d5473e3bbc104e8cf"}
Nov 02 02:07:34.824704 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.824488    4515 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c18b0af6b09a2191d5ef9b3fa3a276d8607c299f83a45d1d5473e3bbc104e8cf"
Nov 02 02:07:34.828339 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.828319    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 02:07:34.969379 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.969354    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-envoy\") pod \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") "
Nov 02 02:07:34.969548 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.969417    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-data\") pod \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") "
Nov 02 02:07:34.969548 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.969444    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istiod-ca-cert\") pod \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") "
Nov 02 02:07:34.969548 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.969473    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-workload-certs\") pod \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") "
Nov 02 02:07:34.969548 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.969499    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-kube-api-access-c95cv\" (UniqueName: \"kubernetes.io/projected/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-kube-api-access-c95cv\") pod \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") "
Nov 02 02:07:34.969548 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.969518    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-token\" (UniqueName: \"kubernetes.io/projected/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-token\") pod \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") "
Nov 02 02:07:34.969843 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.969577    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-credential-socket\") pod \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") "
Nov 02 02:07:34.969843 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.969674    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-podinfo\") pod \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") "
Nov 02 02:07:34.969843 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.969742    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-data" pod "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" (UID: "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:34.969843 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.969772    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-workload-socket\") pod \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\" (UID: \"fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004\") "
Nov 02 02:07:34.969843 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.969771    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-workload-certs" pod "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" (UID: "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:34.970139 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.969911    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-credential-socket" pod "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" (UID: "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:34.970139 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.969931    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istiod-ca-cert" pod "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" (UID: "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:07:34.970257 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.970150    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-workload-socket" pod "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" (UID: "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:34.970402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.970365    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:34.970402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.970391    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:34.970402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.970405    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:34.970655 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.970419    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:34.970655 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.970433    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:34.971612 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.971584    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-envoy" pod "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" (UID: "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:34.971706 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.971676    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-kube-api-access-c95cv" pod "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" (UID: "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004"). InnerVolumeSpecName "kube-api-access-c95cv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:07:34.971787 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.971760    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-token" pod "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" (UID: "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:07:34.971856 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:34.971839    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-podinfo" pod "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" (UID: "fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:07:35.071490 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:35.071459    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c95cv\" (UniqueName: \"kubernetes.io/projected/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-kube-api-access-c95cv\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:35.071490 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:35.071491    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:35.071677 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:35.071507    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:35.071677 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:35.071521    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:35.829632 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:35.829601    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"
Nov 02 02:07:36.043337 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:36.043305    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"]
Nov 02 02:07:36.061175 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:36.061147    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-ed0600-openshift-default-55768f8df5-2gbfm"]
Nov 02 02:07:36.952560 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:36.952520    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" path="/var/lib/kubelet/pods/fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004/volumes"
Nov 02 02:07:39.958730 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:07:39.958695    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:07:40.336894 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:40.336806    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"]
Nov 02 02:07:40.386228 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:40.386201    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"]
Nov 02 02:07:47.745575 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:47.745523    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"]
Nov 02 02:07:47.748710 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:47.748663    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2" podUID="79ffb456-7b20-4afc-8442-0772cf97b269" containerName="istio-proxy" containerID="cri-o://8f84956af8d7b42b4a9fb9cd9fe3276777ea7d588b4d562eb4885c794467dd7d" gracePeriod=30
Nov 02 02:07:51.406281 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:51.406252    4515 scope.go:122] "RemoveContainer" containerID="23090ea8c8193ff14c7651e63cddc3d9ce2bae01a5790b0263741c74e76a985d"
Nov 02 02:07:52.945618 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:52.945581    4515 generic.go:356] "Generic (PLEG): container finished" podID="79ffb456-7b20-4afc-8442-0772cf97b269" containerID="8f84956af8d7b42b4a9fb9cd9fe3276777ea7d588b4d562eb4885c794467dd7d" exitCode=0
Nov 02 02:07:52.946047 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:52.945651    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2" event={"ID":"79ffb456-7b20-4afc-8442-0772cf97b269","Type":"ContainerDied","Data":"8f84956af8d7b42b4a9fb9cd9fe3276777ea7d588b4d562eb4885c794467dd7d"}
Nov 02 02:07:53.009457 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.009433    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:07:53.160532 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.160460    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79ffb456-7b20-4afc-8442-0772cf97b269-kube-api-access-vvwt8\" (UniqueName: \"kubernetes.io/projected/79ffb456-7b20-4afc-8442-0772cf97b269-kube-api-access-vvwt8\") pod \"79ffb456-7b20-4afc-8442-0772cf97b269\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") "
Nov 02 02:07:53.160681 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.160560    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-istio-envoy\") pod \"79ffb456-7b20-4afc-8442-0772cf97b269\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") "
Nov 02 02:07:53.160681 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.160593    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-workload-certs\") pod \"79ffb456-7b20-4afc-8442-0772cf97b269\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") "
Nov 02 02:07:53.160681 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.160626    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-istio-data\") pod \"79ffb456-7b20-4afc-8442-0772cf97b269\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") "
Nov 02 02:07:53.160681 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.160666    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-credential-socket\") pod \"79ffb456-7b20-4afc-8442-0772cf97b269\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") "
Nov 02 02:07:53.160907 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.160722    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/79ffb456-7b20-4afc-8442-0772cf97b269-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79ffb456-7b20-4afc-8442-0772cf97b269-istio-podinfo\") pod \"79ffb456-7b20-4afc-8442-0772cf97b269\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") "
Nov 02 02:07:53.160907 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.160754    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-workload-socket\") pod \"79ffb456-7b20-4afc-8442-0772cf97b269\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") "
Nov 02 02:07:53.160907 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.160806    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79ffb456-7b20-4afc-8442-0772cf97b269-istio-token\" (UniqueName: \"kubernetes.io/projected/79ffb456-7b20-4afc-8442-0772cf97b269-istio-token\") pod \"79ffb456-7b20-4afc-8442-0772cf97b269\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") "
Nov 02 02:07:53.160907 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.160887    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/79ffb456-7b20-4afc-8442-0772cf97b269-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79ffb456-7b20-4afc-8442-0772cf97b269-istiod-ca-cert\") pod \"79ffb456-7b20-4afc-8442-0772cf97b269\" (UID: \"79ffb456-7b20-4afc-8442-0772cf97b269\") "
Nov 02 02:07:53.161104 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.160890    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-workload-certs" pod "79ffb456-7b20-4afc-8442-0772cf97b269" (UID: "79ffb456-7b20-4afc-8442-0772cf97b269"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:53.161104 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.160904    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-istio-data" pod "79ffb456-7b20-4afc-8442-0772cf97b269" (UID: "79ffb456-7b20-4afc-8442-0772cf97b269"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:53.161104 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.160974    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-credential-socket" pod "79ffb456-7b20-4afc-8442-0772cf97b269" (UID: "79ffb456-7b20-4afc-8442-0772cf97b269"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:53.161326 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.161221    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-workload-socket" pod "79ffb456-7b20-4afc-8442-0772cf97b269" (UID: "79ffb456-7b20-4afc-8442-0772cf97b269"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:53.161380 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.161326    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79ffb456-7b20-4afc-8442-0772cf97b269-istiod-ca-cert" pod "79ffb456-7b20-4afc-8442-0772cf97b269" (UID: "79ffb456-7b20-4afc-8442-0772cf97b269"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:07:53.161424 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.161412    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:53.161471 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.161433    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:53.161471 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.161448    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:53.161471 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.161464    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:53.162766 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.162732    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79ffb456-7b20-4afc-8442-0772cf97b269-kube-api-access-vvwt8" pod "79ffb456-7b20-4afc-8442-0772cf97b269" (UID: "79ffb456-7b20-4afc-8442-0772cf97b269"). InnerVolumeSpecName "kube-api-access-vvwt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:07:53.162883 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.162854    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79ffb456-7b20-4afc-8442-0772cf97b269-istio-token" pod "79ffb456-7b20-4afc-8442-0772cf97b269" (UID: "79ffb456-7b20-4afc-8442-0772cf97b269"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:07:53.163092 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.163076    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-istio-envoy" pod "79ffb456-7b20-4afc-8442-0772cf97b269" (UID: "79ffb456-7b20-4afc-8442-0772cf97b269"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:53.163234 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.163213    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/79ffb456-7b20-4afc-8442-0772cf97b269-istio-podinfo" pod "79ffb456-7b20-4afc-8442-0772cf97b269" (UID: "79ffb456-7b20-4afc-8442-0772cf97b269"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:07:53.262510 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.262472    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79ffb456-7b20-4afc-8442-0772cf97b269-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:53.262510 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.262503    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79ffb456-7b20-4afc-8442-0772cf97b269-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:53.262510 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.262515    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79ffb456-7b20-4afc-8442-0772cf97b269-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:53.262734 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.262528    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvwt8\" (UniqueName: \"kubernetes.io/projected/79ffb456-7b20-4afc-8442-0772cf97b269-kube-api-access-vvwt8\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:53.262734 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.262542    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79ffb456-7b20-4afc-8442-0772cf97b269-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:53.952607 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.952576    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"
Nov 02 02:07:53.952607 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.952587    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2" event={"ID":"79ffb456-7b20-4afc-8442-0772cf97b269","Type":"ContainerDied","Data":"b2a91d76178e0fcc5924155c7692efb8779d7de9018ce50fba0fe265aad354cd"}
Nov 02 02:07:53.953188 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:53.952639    4515 scope.go:122] "RemoveContainer" containerID="8f84956af8d7b42b4a9fb9cd9fe3276777ea7d588b4d562eb4885c794467dd7d"
Nov 02 02:07:54.170396 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:54.170354    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"]
Nov 02 02:07:54.237626 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:54.237538    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c1aad5-openshift-default-6b9469f649-rmfp2"]
Nov 02 02:07:54.956816 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:54.956768    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="79ffb456-7b20-4afc-8442-0772cf97b269" path="/var/lib/kubelet/pods/79ffb456-7b20-4afc-8442-0772cf97b269/volumes"
Nov 02 02:07:55.357819 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.357717    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerName="main" containerID="cri-o://3d3512e4037dec03455287ce170de81be3f473d14e2017ab1d9ac2eed8546756" gracePeriod=45
Nov 02 02:07:55.406665 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.406634    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" podUID="747c597f-77cf-4287-a7a1-879565aec8ae" containerName="main" containerID="cri-o://b1770a23cc5846868b62a1fe28b358cbbc99f9819f31acdeac0b2a7875ec6a1b" gracePeriod=45
Nov 02 02:07:55.406859 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.406836    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": dial tcp 10.133.0.49:8000: connect: connection refused"
Nov 02 02:07:55.668391 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.668360    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 02:07:55.696637 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.696616    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 02:07:55.734003 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.733976    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-dshm\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-dshm\") pod \"747c597f-77cf-4287-a7a1-879565aec8ae\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") "
Nov 02 02:07:55.734133 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.734013    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-tls-certs\") pod \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") "
Nov 02 02:07:55.734133 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.734075    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-home\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-home\") pod \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") "
Nov 02 02:07:55.734133 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.734127    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-home\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-home\") pod \"747c597f-77cf-4287-a7a1-879565aec8ae\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") "
Nov 02 02:07:55.734309 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.734160    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-dshm\") pod \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") "
Nov 02 02:07:55.734309 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.734193    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-model-cache\") pod \"747c597f-77cf-4287-a7a1-879565aec8ae\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") "
Nov 02 02:07:55.734309 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.734250    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-model-cache\") pod \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") "
Nov 02 02:07:55.734309 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.734292    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-tmp-dir\") pod \"747c597f-77cf-4287-a7a1-879565aec8ae\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") "
Nov 02 02:07:55.734503 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.734368    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/747c597f-77cf-4287-a7a1-879565aec8ae-tls-certs\" (UniqueName: \"kubernetes.io/secret/747c597f-77cf-4287-a7a1-879565aec8ae-tls-certs\") pod \"747c597f-77cf-4287-a7a1-879565aec8ae\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") "
Nov 02 02:07:55.734503 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.734415    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-tmp-dir\") pod \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") "
Nov 02 02:07:55.734503 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.734456    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-kube-api-access-8fmxt\" (UniqueName: \"kubernetes.io/projected/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-kube-api-access-8fmxt\") pod \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\" (UID: \"f8e7a4cb-f925-40e2-9968-b8a0e19c283e\") "
Nov 02 02:07:55.734503 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.734488    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/747c597f-77cf-4287-a7a1-879565aec8ae-kube-api-access-lgbxm\" (UniqueName: \"kubernetes.io/projected/747c597f-77cf-4287-a7a1-879565aec8ae-kube-api-access-lgbxm\") pod \"747c597f-77cf-4287-a7a1-879565aec8ae\" (UID: \"747c597f-77cf-4287-a7a1-879565aec8ae\") "
Nov 02 02:07:55.734983 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.734481    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-home" pod "747c597f-77cf-4287-a7a1-879565aec8ae" (UID: "747c597f-77cf-4287-a7a1-879565aec8ae"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:55.735087 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.734491    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-home" pod "f8e7a4cb-f925-40e2-9968-b8a0e19c283e" (UID: "f8e7a4cb-f925-40e2-9968-b8a0e19c283e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:55.735087 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.734717    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-model-cache" pod "747c597f-77cf-4287-a7a1-879565aec8ae" (UID: "747c597f-77cf-4287-a7a1-879565aec8ae"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:55.735087 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.734722    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-tmp-dir" pod "f8e7a4cb-f925-40e2-9968-b8a0e19c283e" (UID: "f8e7a4cb-f925-40e2-9968-b8a0e19c283e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:55.735087 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.734667    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-model-cache" pod "f8e7a4cb-f925-40e2-9968-b8a0e19c283e" (UID: "f8e7a4cb-f925-40e2-9968-b8a0e19c283e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:55.735087 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.734901    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-tmp-dir" pod "747c597f-77cf-4287-a7a1-879565aec8ae" (UID: "747c597f-77cf-4287-a7a1-879565aec8ae"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:55.736537 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.736504    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-dshm" pod "747c597f-77cf-4287-a7a1-879565aec8ae" (UID: "747c597f-77cf-4287-a7a1-879565aec8ae"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:55.736651 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.736627    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-kube-api-access-8fmxt" pod "f8e7a4cb-f925-40e2-9968-b8a0e19c283e" (UID: "f8e7a4cb-f925-40e2-9968-b8a0e19c283e"). InnerVolumeSpecName "kube-api-access-8fmxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:07:55.736718 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.736683    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-dshm" pod "f8e7a4cb-f925-40e2-9968-b8a0e19c283e" (UID: "f8e7a4cb-f925-40e2-9968-b8a0e19c283e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:07:55.736718 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.736701    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-tls-certs" pod "f8e7a4cb-f925-40e2-9968-b8a0e19c283e" (UID: "f8e7a4cb-f925-40e2-9968-b8a0e19c283e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:07:55.737045 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.737025    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/747c597f-77cf-4287-a7a1-879565aec8ae-tls-certs" pod "747c597f-77cf-4287-a7a1-879565aec8ae" (UID: "747c597f-77cf-4287-a7a1-879565aec8ae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:07:55.737211 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.737197    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/747c597f-77cf-4287-a7a1-879565aec8ae-kube-api-access-lgbxm" pod "747c597f-77cf-4287-a7a1-879565aec8ae" (UID: "747c597f-77cf-4287-a7a1-879565aec8ae"). InnerVolumeSpecName "kube-api-access-lgbxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:07:55.836678 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.836638    4515 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/747c597f-77cf-4287-a7a1-879565aec8ae-tls-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:55.836678 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.836678    4515 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-tmp-dir\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:55.836924 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.836695    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fmxt\" (UniqueName: \"kubernetes.io/projected/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-kube-api-access-8fmxt\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:55.836924 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.836711    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgbxm\" (UniqueName: \"kubernetes.io/projected/747c597f-77cf-4287-a7a1-879565aec8ae-kube-api-access-lgbxm\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:55.836924 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.836725    4515 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-dshm\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:55.836924 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.836737    4515 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-tls-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:55.836924 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.836749    4515 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-home\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:55.836924 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.836762    4515 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-home\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:55.836924 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.836774    4515 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-dshm\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:55.836924 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.836786    4515 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-model-cache\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:55.836924 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.836799    4515 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8e7a4cb-f925-40e2-9968-b8a0e19c283e-model-cache\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:55.836924 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.836812    4515 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747c597f-77cf-4287-a7a1-879565aec8ae-tmp-dir\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:07:55.966773 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.966738    4515 generic.go:356] "Generic (PLEG): container finished" podID="747c597f-77cf-4287-a7a1-879565aec8ae" containerID="b1770a23cc5846868b62a1fe28b358cbbc99f9819f31acdeac0b2a7875ec6a1b" exitCode=0
Nov 02 02:07:55.967245 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.966784    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"
Nov 02 02:07:55.967245 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.966819    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" event={"ID":"747c597f-77cf-4287-a7a1-879565aec8ae","Type":"ContainerDied","Data":"b1770a23cc5846868b62a1fe28b358cbbc99f9819f31acdeac0b2a7875ec6a1b"}
Nov 02 02:07:55.967245 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.966862    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s" event={"ID":"747c597f-77cf-4287-a7a1-879565aec8ae","Type":"ContainerDied","Data":"ee236b03be60100cc1154a1150749a33b1326f0ce7bb5562fdbc56162f8eac6a"}
Nov 02 02:07:55.967245 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.966885    4515 scope.go:122] "RemoveContainer" containerID="b1770a23cc5846868b62a1fe28b358cbbc99f9819f31acdeac0b2a7875ec6a1b"
Nov 02 02:07:55.968293 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.968270    4515 generic.go:356] "Generic (PLEG): container finished" podID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerID="3d3512e4037dec03455287ce170de81be3f473d14e2017ab1d9ac2eed8546756" exitCode=0
Nov 02 02:07:55.968381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.968305    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"
Nov 02 02:07:55.968381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.968344    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" event={"ID":"f8e7a4cb-f925-40e2-9968-b8a0e19c283e","Type":"ContainerDied","Data":"3d3512e4037dec03455287ce170de81be3f473d14e2017ab1d9ac2eed8546756"}
Nov 02 02:07:55.968381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.968375    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg" event={"ID":"f8e7a4cb-f925-40e2-9968-b8a0e19c283e","Type":"ContainerDied","Data":"06a2c00e1e7fef34a1412b8d3047bdc8922e494c58a49ce6aeddb4e53fbe8df8"}
Nov 02 02:07:55.976000 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.975985    4515 scope.go:122] "RemoveContainer" containerID="b1770a23cc5846868b62a1fe28b358cbbc99f9819f31acdeac0b2a7875ec6a1b"
Nov 02 02:07:55.976302 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:07:55.976283    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1770a23cc5846868b62a1fe28b358cbbc99f9819f31acdeac0b2a7875ec6a1b\": container with ID starting with b1770a23cc5846868b62a1fe28b358cbbc99f9819f31acdeac0b2a7875ec6a1b not found: ID does not exist" containerID="b1770a23cc5846868b62a1fe28b358cbbc99f9819f31acdeac0b2a7875ec6a1b"
Nov 02 02:07:55.976373 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.976310    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1770a23cc5846868b62a1fe28b358cbbc99f9819f31acdeac0b2a7875ec6a1b"} err="failed to get container status \"b1770a23cc5846868b62a1fe28b358cbbc99f9819f31acdeac0b2a7875ec6a1b\": rpc error: code = NotFound desc = could not find container \"b1770a23cc5846868b62a1fe28b358cbbc99f9819f31acdeac0b2a7875ec6a1b\": container with ID starting with b1770a23cc5846868b62a1fe28b358cbbc99f9819f31acdeac0b2a7875ec6a1b not found: ID does not exist"
Nov 02 02:07:55.976373 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.976327    4515 scope.go:122] "RemoveContainer" containerID="3d3512e4037dec03455287ce170de81be3f473d14e2017ab1d9ac2eed8546756"
Nov 02 02:07:55.984268 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.984252    4515 scope.go:122] "RemoveContainer" containerID="3d3512e4037dec03455287ce170de81be3f473d14e2017ab1d9ac2eed8546756"
Nov 02 02:07:55.984509 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:07:55.984493    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d3512e4037dec03455287ce170de81be3f473d14e2017ab1d9ac2eed8546756\": container with ID starting with 3d3512e4037dec03455287ce170de81be3f473d14e2017ab1d9ac2eed8546756 not found: ID does not exist" containerID="3d3512e4037dec03455287ce170de81be3f473d14e2017ab1d9ac2eed8546756"
Nov 02 02:07:55.984551 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:55.984521    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d3512e4037dec03455287ce170de81be3f473d14e2017ab1d9ac2eed8546756"} err="failed to get container status \"3d3512e4037dec03455287ce170de81be3f473d14e2017ab1d9ac2eed8546756\": rpc error: code = NotFound desc = could not find container \"3d3512e4037dec03455287ce170de81be3f473d14e2017ab1d9ac2eed8546756\": container with ID starting with 3d3512e4037dec03455287ce170de81be3f473d14e2017ab1d9ac2eed8546756 not found: ID does not exist"
Nov 02 02:07:56.160623 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:56.160592    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"]
Nov 02 02:07:56.166208 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:56.166172    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-ed0600/distinct-w4-ed0600-kserve-699d4f9f98-mt4gg"]
Nov 02 02:07:56.287992 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:56.287901    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"]
Nov 02 02:07:56.291759 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:56.291734    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-ed0600/simulator-w4-ed0600-kserve-bf55d8dc4-6d59s"]
Nov 02 02:07:56.956683 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:56.956652    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="747c597f-77cf-4287-a7a1-879565aec8ae" path="/var/lib/kubelet/pods/747c597f-77cf-4287-a7a1-879565aec8ae/volumes"
Nov 02 02:07:56.957144 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:07:56.957110    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" path="/var/lib/kubelet/pods/f8e7a4cb-f925-40e2-9968-b8a0e19c283e/volumes"
Nov 02 02:08:09.958700 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:08:09.958659    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:08:23.668374 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.668342    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"]
Nov 02 02:08:23.668800 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.668785    4515 cpu_manager.go:414] "containerMap: removing container" podUID="79ffb456-7b20-4afc-8442-0772cf97b269" containerName="istio-proxy"
Nov 02 02:08:23.668845 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.668803    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79ffb456-7b20-4afc-8442-0772cf97b269" containerName="istio-proxy"
Nov 02 02:08:23.668845 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.668837    4515 cpu_manager.go:414] "containerMap: removing container" podUID="747c597f-77cf-4287-a7a1-879565aec8ae" containerName="main"
Nov 02 02:08:23.668845 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.668842    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="747c597f-77cf-4287-a7a1-879565aec8ae" containerName="main"
Nov 02 02:08:23.668933 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.668848    4515 cpu_manager.go:414] "containerMap: removing container" podUID="fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" containerName="istio-proxy"
Nov 02 02:08:23.668933 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.668853    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" containerName="istio-proxy"
Nov 02 02:08:23.668933 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.668866    4515 cpu_manager.go:414] "containerMap: removing container" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerName="main"
Nov 02 02:08:23.668933 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.668871    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerName="main"
Nov 02 02:08:23.669050 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.668940    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="79ffb456-7b20-4afc-8442-0772cf97b269" containerName="istio-proxy"
Nov 02 02:08:23.669050 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.668957    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="747c597f-77cf-4287-a7a1-879565aec8ae" containerName="main"
Nov 02 02:08:23.669050 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.668966    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd2855c3-4dfa-4b84-a7b2-7bc7dd9b4004" containerName="istio-proxy"
Nov 02 02:08:23.669050 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.668973    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8e7a4cb-f925-40e2-9968-b8a0e19c283e" containerName="main"
Nov 02 02:08:23.672153 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.672133    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.726759 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.726728    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5700d3d1-openshift-default-dockercfg-z5s4p\""
Nov 02 02:08:23.776809 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.776777    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"]
Nov 02 02:08:23.825436 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.825409    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkxcx\" (UniqueName: \"kubernetes.io/projected/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-kube-api-access-jkxcx\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.825666 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.825442    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-workload-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.825666 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.825547    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-workload-certs\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.825666 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.825613    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-podinfo\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.825666 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.825634    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istiod-ca-cert\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.825823 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.825726    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-credential-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.825823 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.825755    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-envoy\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.825823 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.825792    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-token\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.825823 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.825812    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-data\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.926794 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.926719    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkxcx\" (UniqueName: \"kubernetes.io/projected/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-kube-api-access-jkxcx\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.926794 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.926755    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-workload-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.927001 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.926803    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-workload-certs\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.927001 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.926835    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-podinfo\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.927001 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.926852    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istiod-ca-cert\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.927001 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.926918    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-credential-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.927001 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.926948    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-envoy\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.927001 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.926998    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-token\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.927375 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.927029    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-data\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.927375 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.927291    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-workload-certs\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.927375 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.927356    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-workload-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.927490 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.927440    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-credential-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.927611 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.927587    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-data\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.927740 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.927660    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istiod-ca-cert\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.929288 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.929268    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-envoy\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.929510 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.929493    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-podinfo\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.987420 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.987387    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-token\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:23.987540 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:23.987457    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkxcx\" (UniqueName: \"kubernetes.io/projected/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-kube-api-access-jkxcx\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:24.285270 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:24.285234    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:24.716509 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:24.716479    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"]
Nov 02 02:08:24.718687 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:08:24.718661    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc70d92a5_6ed0_470d_9c5d_822ffc97bbbc.slice/crio-eca9b584f6f7248a43f8823d371cf16bb60d95195113d71699b5ea2fc26d3dd4 WatchSource:0}: Error finding container eca9b584f6f7248a43f8823d371cf16bb60d95195113d71699b5ea2fc26d3dd4: Status 404 returned error can't find the container with id eca9b584f6f7248a43f8823d371cf16bb60d95195113d71699b5ea2fc26d3dd4
Nov 02 02:08:24.721061 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:24.721026    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:08:24.721154 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:24.721097    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:08:24.721154 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:24.721145    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:08:25.223523 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:25.223488    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp" event={"ID":"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc","Type":"ContainerStarted","Data":"0d7e71131516091164d9a188442bd4851c7653b291cc67530ccedf88602e3044"}
Nov 02 02:08:25.223694 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:25.223544    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp" event={"ID":"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc","Type":"ContainerStarted","Data":"eca9b584f6f7248a43f8823d371cf16bb60d95195113d71699b5ea2fc26d3dd4"}
Nov 02 02:08:25.285641 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:25.285609    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:25.286964 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:25.286937    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 02 02:08:25.287091 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:25.286994    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp" podUID="c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 02 02:08:25.394590 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:25.394537    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp" podStartSLOduration=2.394522896 podStartE2EDuration="2.394522896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:08:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:08:25.391338463 +0000 UTC m=+2194.603708273" watchObservedRunningTime="2026-11-02 02:08:25.394522896 +0000 UTC m=+2194.606892703"
Nov 02 02:08:26.286454 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:26.286426    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 02 02:08:26.286831 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:26.286473    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp" podUID="c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 02 02:08:27.286319 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:27.286283    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 02 02:08:27.286511 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:27.286342    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp" podUID="c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 02 02:08:28.285973 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:28.285934    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 02 02:08:28.286186 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:28.286006    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp" podUID="c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 02 02:08:29.285656 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:29.285624    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 02 02:08:29.286192 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:29.285682    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp" podUID="c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 02 02:08:30.285813 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:30.285775    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 02 02:08:30.286271 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:30.285827    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp" podUID="c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 02 02:08:31.286267 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:31.286218    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 02 02:08:31.286648 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:31.286299    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp" podUID="c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 02 02:08:32.290162 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:32.290112    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:32.290541 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:32.290402    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:32.291196 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:32.291177    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:08:39.983551 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:08:39.983403    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:08:43.374767 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:43.374683    4515 generic.go:356] "Generic (PLEG): container finished" podID="7bcde5e3-f9bf-4724-8a21-c6a63c13df32" containerID="8a17a1bc19f12cee6a79978f19371b61dfa0d865648117fe4c5f2390ddff86e5" exitCode=0
Nov 02 02:08:43.374767 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:43.374755    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh" event={"ID":"7bcde5e3-f9bf-4724-8a21-c6a63c13df32","Type":"ContainerDied","Data":"8a17a1bc19f12cee6a79978f19371b61dfa0d865648117fe4c5f2390ddff86e5"}
Nov 02 02:08:43.375275 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:43.375207    4515 scope.go:122] "RemoveContainer" containerID="8a17a1bc19f12cee6a79978f19371b61dfa0d865648117fe4c5f2390ddff86e5"
Nov 02 02:08:43.376745 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:43.376714    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:08:43.376838 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:43.376782    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:08:43.376838 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:43.376814    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:08:44.417145 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:44.414975    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh" event={"ID":"7bcde5e3-f9bf-4724-8a21-c6a63c13df32","Type":"ContainerStarted","Data":"c7cc62ba795694a107e63bf8c8e7cc193c09b44a10ff4cb3b768e818ad3de856"}
Nov 02 02:08:45.018756 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:45.018725    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:08:45.018944 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:45.018903    4515 patch_prober.go:29] interesting pod/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 02 02:08:45.018989 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:45.018950    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh" podUID="7bcde5e3-f9bf-4724-8a21-c6a63c13df32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 02 02:08:46.019950 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:46.019915    4515 patch_prober.go:29] interesting pod/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 02 02:08:46.020380 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:46.019974    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh" podUID="7bcde5e3-f9bf-4724-8a21-c6a63c13df32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 02 02:08:47.019676 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:47.019640    4515 patch_prober.go:29] interesting pod/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 02 02:08:47.019859 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:47.019705    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh" podUID="7bcde5e3-f9bf-4724-8a21-c6a63c13df32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 02 02:08:48.018961 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:48.018917    4515 patch_prober.go:29] interesting pod/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 02 02:08:48.019378 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:48.018983    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh" podUID="7bcde5e3-f9bf-4724-8a21-c6a63c13df32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 02 02:08:49.019770 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:49.019738    4515 patch_prober.go:29] interesting pod/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 02 02:08:49.020209 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:49.019788    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh" podUID="7bcde5e3-f9bf-4724-8a21-c6a63c13df32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 02 02:08:50.023509 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:50.023480    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:08:50.023912 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:50.023891    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:08:50.024564 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:50.024545    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:08:53.359409 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:53.359374    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"]
Nov 02 02:08:53.359901 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:53.359625    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz" podUID="4517f292-db6c-4474-9162-0d22eeda607a" containerName="istio-proxy" containerID="cri-o://00d4067d18816c9a8b9cd54df95472267324647c37d520e540aab223778ecf75" gracePeriod=30
Nov 02 02:08:57.150329 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:57.150291    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"]
Nov 02 02:08:57.163020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:57.162980    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"]
Nov 02 02:08:57.174703 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:57.174673    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"]
Nov 02 02:08:58.073507 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.073467    4515 patch_prober.go:29] interesting pod/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 02:08:58.073686 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.073519    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz" podUID="4517f292-db6c-4474-9162-0d22eeda607a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 02:08:58.548615 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.548426    4515 generic.go:356] "Generic (PLEG): container finished" podID="4517f292-db6c-4474-9162-0d22eeda607a" containerID="00d4067d18816c9a8b9cd54df95472267324647c37d520e540aab223778ecf75" exitCode=0
Nov 02 02:08:58.548615 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.548541    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz" event={"ID":"4517f292-db6c-4474-9162-0d22eeda607a","Type":"ContainerDied","Data":"00d4067d18816c9a8b9cd54df95472267324647c37d520e540aab223778ecf75"}
Nov 02 02:08:58.627155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.627065    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 02:08:58.764217 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.764181    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4517f292-db6c-4474-9162-0d22eeda607a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4517f292-db6c-4474-9162-0d22eeda607a-istiod-ca-cert\") pod \"4517f292-db6c-4474-9162-0d22eeda607a\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") "
Nov 02 02:08:58.764406 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.764226    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-istio-data\") pod \"4517f292-db6c-4474-9162-0d22eeda607a\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") "
Nov 02 02:08:58.764406 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.764249    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-istio-envoy\") pod \"4517f292-db6c-4474-9162-0d22eeda607a\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") "
Nov 02 02:08:58.764406 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.764271    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-workload-certs\") pod \"4517f292-db6c-4474-9162-0d22eeda607a\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") "
Nov 02 02:08:58.764406 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.764307    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4517f292-db6c-4474-9162-0d22eeda607a-kube-api-access-647vl\" (UniqueName: \"kubernetes.io/projected/4517f292-db6c-4474-9162-0d22eeda607a-kube-api-access-647vl\") pod \"4517f292-db6c-4474-9162-0d22eeda607a\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") "
Nov 02 02:08:58.764406 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.764361    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-workload-socket\") pod \"4517f292-db6c-4474-9162-0d22eeda607a\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") "
Nov 02 02:08:58.764698 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.764424    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-credential-socket\") pod \"4517f292-db6c-4474-9162-0d22eeda607a\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") "
Nov 02 02:08:58.764698 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.764453    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4517f292-db6c-4474-9162-0d22eeda607a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4517f292-db6c-4474-9162-0d22eeda607a-istio-podinfo\") pod \"4517f292-db6c-4474-9162-0d22eeda607a\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") "
Nov 02 02:08:58.764698 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.764510    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4517f292-db6c-4474-9162-0d22eeda607a-istio-token\" (UniqueName: \"kubernetes.io/projected/4517f292-db6c-4474-9162-0d22eeda607a-istio-token\") pod \"4517f292-db6c-4474-9162-0d22eeda607a\" (UID: \"4517f292-db6c-4474-9162-0d22eeda607a\") "
Nov 02 02:08:58.764698 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.764579    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-istio-data" pod "4517f292-db6c-4474-9162-0d22eeda607a" (UID: "4517f292-db6c-4474-9162-0d22eeda607a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:08:58.764698 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.764609    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4517f292-db6c-4474-9162-0d22eeda607a-istiod-ca-cert" pod "4517f292-db6c-4474-9162-0d22eeda607a" (UID: "4517f292-db6c-4474-9162-0d22eeda607a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:08:58.764964 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.764737    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-workload-socket" pod "4517f292-db6c-4474-9162-0d22eeda607a" (UID: "4517f292-db6c-4474-9162-0d22eeda607a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:08:58.764964 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.764750    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-credential-socket" pod "4517f292-db6c-4474-9162-0d22eeda607a" (UID: "4517f292-db6c-4474-9162-0d22eeda607a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:08:58.764964 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.764850    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-workload-certs" pod "4517f292-db6c-4474-9162-0d22eeda607a" (UID: "4517f292-db6c-4474-9162-0d22eeda607a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:08:58.765159 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.765109    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:08:58.765221 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.765168    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4517f292-db6c-4474-9162-0d22eeda607a-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:08:58.765221 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.765183    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:08:58.765221 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.765196    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:08:58.765221 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.765210    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:08:58.766702 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.766671    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-istio-envoy" pod "4517f292-db6c-4474-9162-0d22eeda607a" (UID: "4517f292-db6c-4474-9162-0d22eeda607a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:08:58.766702 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.766678    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4517f292-db6c-4474-9162-0d22eeda607a-istio-token" pod "4517f292-db6c-4474-9162-0d22eeda607a" (UID: "4517f292-db6c-4474-9162-0d22eeda607a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:08:58.767003 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.766978    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4517f292-db6c-4474-9162-0d22eeda607a-istio-podinfo" pod "4517f292-db6c-4474-9162-0d22eeda607a" (UID: "4517f292-db6c-4474-9162-0d22eeda607a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:08:58.767217 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.767194    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4517f292-db6c-4474-9162-0d22eeda607a-kube-api-access-647vl" pod "4517f292-db6c-4474-9162-0d22eeda607a" (UID: "4517f292-db6c-4474-9162-0d22eeda607a"). InnerVolumeSpecName "kube-api-access-647vl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:08:58.865984 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.865951    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4517f292-db6c-4474-9162-0d22eeda607a-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:08:58.865984 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.865979    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-647vl\" (UniqueName: \"kubernetes.io/projected/4517f292-db6c-4474-9162-0d22eeda607a-kube-api-access-647vl\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:08:58.865984 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.865989    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4517f292-db6c-4474-9162-0d22eeda607a-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:08:58.866240 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:58.865998    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4517f292-db6c-4474-9162-0d22eeda607a-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:08:59.556868 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:59.556831    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz" event={"ID":"4517f292-db6c-4474-9162-0d22eeda607a","Type":"ContainerDied","Data":"686d10c36b0935f077bff1a845f24e854cdb9a7b5709a993b90b62787219f141"}
Nov 02 02:08:59.557370 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:59.556907    4515 scope.go:122] "RemoveContainer" containerID="00d4067d18816c9a8b9cd54df95472267324647c37d520e540aab223778ecf75"
Nov 02 02:08:59.557370 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:59.556843    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"
Nov 02 02:08:59.724195 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:59.724163    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"]
Nov 02 02:08:59.773701 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:08:59.773665    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-9j2dz"]
Nov 02 02:09:00.953993 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:00.953952    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4517f292-db6c-4474-9162-0d22eeda607a" path="/var/lib/kubelet/pods/4517f292-db6c-4474-9162-0d22eeda607a/volumes"
Nov 02 02:09:09.958895 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:09:09.958859    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:09:12.184450 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.184387    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main" containerID="cri-o://4972f69a550bea27d5c832d23ed6cd06fea22f142ab9170e954276b1bd756b14" gracePeriod=45
Nov 02 02:09:12.195901 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.195869    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerName="main" containerID="cri-o://f3c7e82b076b05d0d351f2baef21136876a146fc4529be88af9b9bfcf5c295bc" gracePeriod=45
Nov 02 02:09:12.339335 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.339284    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": dial tcp 10.133.0.42:8000: connect: connection refused"
Nov 02 02:09:12.460043 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.460017    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 02:09:12.490980 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.490954    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 02:09:12.546743 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.546709    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-tmp-dir\") pod \"b1893195-677a-4c3d-8058-7623c4505f3d\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") "
Nov 02 02:09:12.546915 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.546791    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-model-cache\") pod \"b1893195-677a-4c3d-8058-7623c4505f3d\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") "
Nov 02 02:09:12.546915 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.546849    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-home\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-home\") pod \"b1893195-677a-4c3d-8058-7623c4505f3d\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") "
Nov 02 02:09:12.546987 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.546918    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-dshm\") pod \"b1893195-677a-4c3d-8058-7623c4505f3d\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") "
Nov 02 02:09:12.547038 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.546976    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-tmp-dir" pod "b1893195-677a-4c3d-8058-7623c4505f3d" (UID: "b1893195-677a-4c3d-8058-7623c4505f3d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:09:12.547084 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.547051    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b1893195-677a-4c3d-8058-7623c4505f3d-tls-certs\" (UniqueName: \"kubernetes.io/secret/b1893195-677a-4c3d-8058-7623c4505f3d-tls-certs\") pod \"b1893195-677a-4c3d-8058-7623c4505f3d\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") "
Nov 02 02:09:12.547163 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.547082    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-model-cache" pod "b1893195-677a-4c3d-8058-7623c4505f3d" (UID: "b1893195-677a-4c3d-8058-7623c4505f3d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:09:12.547163 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.547137    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-home" pod "b1893195-677a-4c3d-8058-7623c4505f3d" (UID: "b1893195-677a-4c3d-8058-7623c4505f3d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:09:12.547238 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.547162    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1893195-677a-4c3d-8058-7623c4505f3d-kube-api-access-vt5mv\" (UniqueName: \"kubernetes.io/projected/b1893195-677a-4c3d-8058-7623c4505f3d-kube-api-access-vt5mv\") pod \"b1893195-677a-4c3d-8058-7623c4505f3d\" (UID: \"b1893195-677a-4c3d-8058-7623c4505f3d\") "
Nov 02 02:09:12.547584 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.547562    4515 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-tmp-dir\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:12.547584 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.547586    4515 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-model-cache\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:12.547757 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.547599    4515 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-home\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:12.549186 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.549163    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-dshm" pod "b1893195-677a-4c3d-8058-7623c4505f3d" (UID: "b1893195-677a-4c3d-8058-7623c4505f3d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:09:12.549283 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.549221    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1893195-677a-4c3d-8058-7623c4505f3d-tls-certs" pod "b1893195-677a-4c3d-8058-7623c4505f3d" (UID: "b1893195-677a-4c3d-8058-7623c4505f3d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:09:12.549283 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.549262    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1893195-677a-4c3d-8058-7623c4505f3d-kube-api-access-vt5mv" pod "b1893195-677a-4c3d-8058-7623c4505f3d" (UID: "b1893195-677a-4c3d-8058-7623c4505f3d"). InnerVolumeSpecName "kube-api-access-vt5mv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:09:12.583541 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.583493    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" podUID="301ce5d3-728b-4edf-86d4-65a8ea792055" containerName="main" containerID="cri-o://c7c6508e568ab597254acca3fdada287b377ff3df31e1a4a22e3c8884c14952c" gracePeriod=45
Nov 02 02:09:12.647987 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.647958    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-home\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-home\") pod \"6f21efa8-1bf5-449f-addd-10e78c8732dc\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") "
Nov 02 02:09:12.648182 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.648011    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-dshm\") pod \"6f21efa8-1bf5-449f-addd-10e78c8732dc\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") "
Nov 02 02:09:12.648182 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.648073    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-tmp-dir\") pod \"6f21efa8-1bf5-449f-addd-10e78c8732dc\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") "
Nov 02 02:09:12.648182 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.648133    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6f21efa8-1bf5-449f-addd-10e78c8732dc-tls-certs\" (UniqueName: \"kubernetes.io/secret/6f21efa8-1bf5-449f-addd-10e78c8732dc-tls-certs\") pod \"6f21efa8-1bf5-449f-addd-10e78c8732dc\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") "
Nov 02 02:09:12.648363 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.648237    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-model-cache\") pod \"6f21efa8-1bf5-449f-addd-10e78c8732dc\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") "
Nov 02 02:09:12.648363 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.648319    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f21efa8-1bf5-449f-addd-10e78c8732dc-kube-api-access-fp4lj\" (UniqueName: \"kubernetes.io/projected/6f21efa8-1bf5-449f-addd-10e78c8732dc-kube-api-access-fp4lj\") pod \"6f21efa8-1bf5-449f-addd-10e78c8732dc\" (UID: \"6f21efa8-1bf5-449f-addd-10e78c8732dc\") "
Nov 02 02:09:12.648363 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.648318    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-home" pod "6f21efa8-1bf5-449f-addd-10e78c8732dc" (UID: "6f21efa8-1bf5-449f-addd-10e78c8732dc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:09:12.648545 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.648531    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-model-cache" pod "6f21efa8-1bf5-449f-addd-10e78c8732dc" (UID: "6f21efa8-1bf5-449f-addd-10e78c8732dc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:09:12.648724 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.648683    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-tmp-dir" pod "6f21efa8-1bf5-449f-addd-10e78c8732dc" (UID: "6f21efa8-1bf5-449f-addd-10e78c8732dc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:09:12.648929 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.648867    4515 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1893195-677a-4c3d-8058-7623c4505f3d-dshm\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:12.648929 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.648889    4515 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-model-cache\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:12.648929 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.648904    4515 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1893195-677a-4c3d-8058-7623c4505f3d-tls-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:12.648929 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.648916    4515 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-home\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:12.648929 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.648930    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vt5mv\" (UniqueName: \"kubernetes.io/projected/b1893195-677a-4c3d-8058-7623c4505f3d-kube-api-access-vt5mv\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:12.649314 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.648943    4515 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-tmp-dir\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:12.650401 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.650377    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-dshm" pod "6f21efa8-1bf5-449f-addd-10e78c8732dc" (UID: "6f21efa8-1bf5-449f-addd-10e78c8732dc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:09:12.650517 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.650491    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f21efa8-1bf5-449f-addd-10e78c8732dc-tls-certs" pod "6f21efa8-1bf5-449f-addd-10e78c8732dc" (UID: "6f21efa8-1bf5-449f-addd-10e78c8732dc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:09:12.650706 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.650684    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f21efa8-1bf5-449f-addd-10e78c8732dc-kube-api-access-fp4lj" pod "6f21efa8-1bf5-449f-addd-10e78c8732dc" (UID: "6f21efa8-1bf5-449f-addd-10e78c8732dc"). InnerVolumeSpecName "kube-api-access-fp4lj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:09:12.657671 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.657640    4515 generic.go:356] "Generic (PLEG): container finished" podID="b1893195-677a-4c3d-8058-7623c4505f3d" containerID="4972f69a550bea27d5c832d23ed6cd06fea22f142ab9170e954276b1bd756b14" exitCode=0
Nov 02 02:09:12.657802 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.657690    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"
Nov 02 02:09:12.657802 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.657719    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" event={"ID":"b1893195-677a-4c3d-8058-7623c4505f3d","Type":"ContainerDied","Data":"4972f69a550bea27d5c832d23ed6cd06fea22f142ab9170e954276b1bd756b14"}
Nov 02 02:09:12.657802 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.657767    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz" event={"ID":"b1893195-677a-4c3d-8058-7623c4505f3d","Type":"ContainerDied","Data":"7669e7540f561b7fb5799ca0c9116597f1d43e19e54c1f3636d5c2f938000d2d"}
Nov 02 02:09:12.657802 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.657786    4515 scope.go:122] "RemoveContainer" containerID="4972f69a550bea27d5c832d23ed6cd06fea22f142ab9170e954276b1bd756b14"
Nov 02 02:09:12.659255 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.659230    4515 generic.go:356] "Generic (PLEG): container finished" podID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerID="f3c7e82b076b05d0d351f2baef21136876a146fc4529be88af9b9bfcf5c295bc" exitCode=0
Nov 02 02:09:12.659389 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.659275    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"
Nov 02 02:09:12.659389 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.659276    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" event={"ID":"6f21efa8-1bf5-449f-addd-10e78c8732dc","Type":"ContainerDied","Data":"f3c7e82b076b05d0d351f2baef21136876a146fc4529be88af9b9bfcf5c295bc"}
Nov 02 02:09:12.659389 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.659382    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt" event={"ID":"6f21efa8-1bf5-449f-addd-10e78c8732dc","Type":"ContainerDied","Data":"d5186d5680c80712be254ee813236e549e8b67ca6706cd1f80d3736b7e84a8d3"}
Nov 02 02:09:12.709385 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.709081    4515 scope.go:122] "RemoveContainer" containerID="4972f69a550bea27d5c832d23ed6cd06fea22f142ab9170e954276b1bd756b14"
Nov 02 02:09:12.709633 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:09:12.709595    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4972f69a550bea27d5c832d23ed6cd06fea22f142ab9170e954276b1bd756b14\": container with ID starting with 4972f69a550bea27d5c832d23ed6cd06fea22f142ab9170e954276b1bd756b14 not found: ID does not exist" containerID="4972f69a550bea27d5c832d23ed6cd06fea22f142ab9170e954276b1bd756b14"
Nov 02 02:09:12.709706 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.709635    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4972f69a550bea27d5c832d23ed6cd06fea22f142ab9170e954276b1bd756b14"} err="failed to get container status \"4972f69a550bea27d5c832d23ed6cd06fea22f142ab9170e954276b1bd756b14\": rpc error: code = NotFound desc = could not find container \"4972f69a550bea27d5c832d23ed6cd06fea22f142ab9170e954276b1bd756b14\": container with ID starting with 4972f69a550bea27d5c832d23ed6cd06fea22f142ab9170e954276b1bd756b14 not found: ID does not exist"
Nov 02 02:09:12.709706 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.709661    4515 scope.go:122] "RemoveContainer" containerID="f3c7e82b076b05d0d351f2baef21136876a146fc4529be88af9b9bfcf5c295bc"
Nov 02 02:09:12.723163 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.723097    4515 scope.go:122] "RemoveContainer" containerID="f3c7e82b076b05d0d351f2baef21136876a146fc4529be88af9b9bfcf5c295bc"
Nov 02 02:09:12.723545 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:09:12.723513    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3c7e82b076b05d0d351f2baef21136876a146fc4529be88af9b9bfcf5c295bc\": container with ID starting with f3c7e82b076b05d0d351f2baef21136876a146fc4529be88af9b9bfcf5c295bc not found: ID does not exist" containerID="f3c7e82b076b05d0d351f2baef21136876a146fc4529be88af9b9bfcf5c295bc"
Nov 02 02:09:12.723606 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.723550    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3c7e82b076b05d0d351f2baef21136876a146fc4529be88af9b9bfcf5c295bc"} err="failed to get container status \"f3c7e82b076b05d0d351f2baef21136876a146fc4529be88af9b9bfcf5c295bc\": rpc error: code = NotFound desc = could not find container \"f3c7e82b076b05d0d351f2baef21136876a146fc4529be88af9b9bfcf5c295bc\": container with ID starting with f3c7e82b076b05d0d351f2baef21136876a146fc4529be88af9b9bfcf5c295bc not found: ID does not exist"
Nov 02 02:09:12.750364 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.750325    4515 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f21efa8-1bf5-449f-addd-10e78c8732dc-dshm\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:12.750364 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.750362    4515 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f21efa8-1bf5-449f-addd-10e78c8732dc-tls-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:12.750617 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.750377    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fp4lj\" (UniqueName: \"kubernetes.io/projected/6f21efa8-1bf5-449f-addd-10e78c8732dc-kube-api-access-fp4lj\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:12.820952 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.820904    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"]
Nov 02 02:09:12.823820 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.823789    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/simulator-w5-55bc84-kserve-69cd684c9f-dqgzt"]
Nov 02 02:09:12.864351 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.864321    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 02:09:12.931155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.931102    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"]
Nov 02 02:09:12.935323 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.935295    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/premium-w5-55bc84-kserve-69db7c486-xbcvz"]
Nov 02 02:09:12.953154 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.953131    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/301ce5d3-728b-4edf-86d4-65a8ea792055-kube-api-access-8hx6h\" (UniqueName: \"kubernetes.io/projected/301ce5d3-728b-4edf-86d4-65a8ea792055-kube-api-access-8hx6h\") pod \"301ce5d3-728b-4edf-86d4-65a8ea792055\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") "
Nov 02 02:09:12.953326 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.953196    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-model-cache\") pod \"301ce5d3-728b-4edf-86d4-65a8ea792055\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") "
Nov 02 02:09:12.953326 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.953206    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" path="/var/lib/kubelet/pods/6f21efa8-1bf5-449f-addd-10e78c8732dc/volumes"
Nov 02 02:09:12.953326 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.953278    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-tmp-dir\") pod \"301ce5d3-728b-4edf-86d4-65a8ea792055\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") "
Nov 02 02:09:12.953326 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.953304    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-dshm\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-dshm\") pod \"301ce5d3-728b-4edf-86d4-65a8ea792055\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") "
Nov 02 02:09:12.953569 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.953361    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-home\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-home\") pod \"301ce5d3-728b-4edf-86d4-65a8ea792055\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") "
Nov 02 02:09:12.953569 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.953512    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-model-cache" pod "301ce5d3-728b-4edf-86d4-65a8ea792055" (UID: "301ce5d3-728b-4edf-86d4-65a8ea792055"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:09:12.953569 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.953523    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/301ce5d3-728b-4edf-86d4-65a8ea792055-tls-certs\" (UniqueName: \"kubernetes.io/secret/301ce5d3-728b-4edf-86d4-65a8ea792055-tls-certs\") pod \"301ce5d3-728b-4edf-86d4-65a8ea792055\" (UID: \"301ce5d3-728b-4edf-86d4-65a8ea792055\") "
Nov 02 02:09:12.953724 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.953611    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-tmp-dir" pod "301ce5d3-728b-4edf-86d4-65a8ea792055" (UID: "301ce5d3-728b-4edf-86d4-65a8ea792055"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:09:12.953724 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.953641    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-home" pod "301ce5d3-728b-4edf-86d4-65a8ea792055" (UID: "301ce5d3-728b-4edf-86d4-65a8ea792055"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:09:12.953958 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.953937    4515 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-model-cache\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:12.954034 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.953965    4515 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-tmp-dir\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:12.954034 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.953982    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" path="/var/lib/kubelet/pods/b1893195-677a-4c3d-8058-7623c4505f3d/volumes"
Nov 02 02:09:12.954179 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.953980    4515 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-home\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:12.955721 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.955695    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/301ce5d3-728b-4edf-86d4-65a8ea792055-tls-certs" pod "301ce5d3-728b-4edf-86d4-65a8ea792055" (UID: "301ce5d3-728b-4edf-86d4-65a8ea792055"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:09:12.956033 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.956009    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-dshm" pod "301ce5d3-728b-4edf-86d4-65a8ea792055" (UID: "301ce5d3-728b-4edf-86d4-65a8ea792055"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:09:12.956167 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:12.956040    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/301ce5d3-728b-4edf-86d4-65a8ea792055-kube-api-access-8hx6h" pod "301ce5d3-728b-4edf-86d4-65a8ea792055" (UID: "301ce5d3-728b-4edf-86d4-65a8ea792055"). InnerVolumeSpecName "kube-api-access-8hx6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:09:13.065141 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:13.060761    4515 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/301ce5d3-728b-4edf-86d4-65a8ea792055-tls-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:13.065141 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:13.060800    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hx6h\" (UniqueName: \"kubernetes.io/projected/301ce5d3-728b-4edf-86d4-65a8ea792055-kube-api-access-8hx6h\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:13.065141 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:13.060816    4515 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/301ce5d3-728b-4edf-86d4-65a8ea792055-dshm\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:13.666559 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:13.666468    4515 generic.go:356] "Generic (PLEG): container finished" podID="301ce5d3-728b-4edf-86d4-65a8ea792055" containerID="c7c6508e568ab597254acca3fdada287b377ff3df31e1a4a22e3c8884c14952c" exitCode=0
Nov 02 02:09:13.666559 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:13.666527    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"
Nov 02 02:09:13.667077 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:13.666552    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" event={"ID":"301ce5d3-728b-4edf-86d4-65a8ea792055","Type":"ContainerDied","Data":"c7c6508e568ab597254acca3fdada287b377ff3df31e1a4a22e3c8884c14952c"}
Nov 02 02:09:13.667077 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:13.666600    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx" event={"ID":"301ce5d3-728b-4edf-86d4-65a8ea792055","Type":"ContainerDied","Data":"7bfa22a49ffc6707b41956eb8e2ea9e7d97a86ee7b177e0929cf3efd444c124c"}
Nov 02 02:09:13.667077 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:13.666624    4515 scope.go:122] "RemoveContainer" containerID="c7c6508e568ab597254acca3fdada287b377ff3df31e1a4a22e3c8884c14952c"
Nov 02 02:09:13.676981 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:13.676963    4515 scope.go:122] "RemoveContainer" containerID="c7c6508e568ab597254acca3fdada287b377ff3df31e1a4a22e3c8884c14952c"
Nov 02 02:09:13.677289 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:09:13.677268    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7c6508e568ab597254acca3fdada287b377ff3df31e1a4a22e3c8884c14952c\": container with ID starting with c7c6508e568ab597254acca3fdada287b377ff3df31e1a4a22e3c8884c14952c not found: ID does not exist" containerID="c7c6508e568ab597254acca3fdada287b377ff3df31e1a4a22e3c8884c14952c"
Nov 02 02:09:13.677363 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:13.677299    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7c6508e568ab597254acca3fdada287b377ff3df31e1a4a22e3c8884c14952c"} err="failed to get container status \"c7c6508e568ab597254acca3fdada287b377ff3df31e1a4a22e3c8884c14952c\": rpc error: code = NotFound desc = could not find container \"c7c6508e568ab597254acca3fdada287b377ff3df31e1a4a22e3c8884c14952c\": container with ID starting with c7c6508e568ab597254acca3fdada287b377ff3df31e1a4a22e3c8884c14952c not found: ID does not exist"
Nov 02 02:09:13.830261 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:13.830221    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"]
Nov 02 02:09:13.836159 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:13.836129    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-55bc84/distinct-2-w5-55bc84-kserve-67cf6b9b58-tnrkx"]
Nov 02 02:09:14.953267 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:14.953226    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="301ce5d3-728b-4edf-86d4-65a8ea792055" path="/var/lib/kubelet/pods/301ce5d3-728b-4edf-86d4-65a8ea792055/volumes"
Nov 02 02:09:39.976375 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:09:39.976320    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:09:49.805879 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.805845    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"]
Nov 02 02:09:49.806320 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.806296    4515 cpu_manager.go:414] "containerMap: removing container" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerName="main"
Nov 02 02:09:49.806320 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.806309    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerName="main"
Nov 02 02:09:49.806393 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.806326    4515 cpu_manager.go:414] "containerMap: removing container" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main"
Nov 02 02:09:49.806393 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.806331    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main"
Nov 02 02:09:49.806393 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.806339    4515 cpu_manager.go:414] "containerMap: removing container" podUID="301ce5d3-728b-4edf-86d4-65a8ea792055" containerName="main"
Nov 02 02:09:49.806393 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.806344    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="301ce5d3-728b-4edf-86d4-65a8ea792055" containerName="main"
Nov 02 02:09:49.806393 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.806359    4515 cpu_manager.go:414] "containerMap: removing container" podUID="4517f292-db6c-4474-9162-0d22eeda607a" containerName="istio-proxy"
Nov 02 02:09:49.806393 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.806367    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4517f292-db6c-4474-9162-0d22eeda607a" containerName="istio-proxy"
Nov 02 02:09:49.806568 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.806452    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f21efa8-1bf5-449f-addd-10e78c8732dc" containerName="main"
Nov 02 02:09:49.806568 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.806462    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1893195-677a-4c3d-8058-7623c4505f3d" containerName="main"
Nov 02 02:09:49.806568 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.806468    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="301ce5d3-728b-4edf-86d4-65a8ea792055" containerName="main"
Nov 02 02:09:49.806568 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.806474    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="4517f292-db6c-4474-9162-0d22eeda607a" containerName="istio-proxy"
Nov 02 02:09:49.809395 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.809371    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:49.915789 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.915579    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"]
Nov 02 02:09:49.998480 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.998432    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/612f51d4-bab6-4b28-b837-094c15c5166f-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:49.998637 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.998550    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bz8z\" (UniqueName: \"kubernetes.io/projected/612f51d4-bab6-4b28-b837-094c15c5166f-kube-api-access-6bz8z\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:49.998637 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.998621    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/612f51d4-bab6-4b28-b837-094c15c5166f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:49.998741 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.998645    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/612f51d4-bab6-4b28-b837-094c15c5166f-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:49.998859 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.998838    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/612f51d4-bab6-4b28-b837-094c15c5166f-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:49.998903 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.998883    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/612f51d4-bab6-4b28-b837-094c15c5166f-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:49.998989 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.998968    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/612f51d4-bab6-4b28-b837-094c15c5166f-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:49.999051 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.999006    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/612f51d4-bab6-4b28-b837-094c15c5166f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:49.999095 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:49.999044    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/612f51d4-bab6-4b28-b837-094c15c5166f-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.100288 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.100187    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/612f51d4-bab6-4b28-b837-094c15c5166f-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.100288 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.100254    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/612f51d4-bab6-4b28-b837-094c15c5166f-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.100288 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.100282    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/612f51d4-bab6-4b28-b837-094c15c5166f-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.100570 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.100305    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/612f51d4-bab6-4b28-b837-094c15c5166f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.100570 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.100336    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/612f51d4-bab6-4b28-b837-094c15c5166f-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.100570 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.100433    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/612f51d4-bab6-4b28-b837-094c15c5166f-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.100570 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.100487    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bz8z\" (UniqueName: \"kubernetes.io/projected/612f51d4-bab6-4b28-b837-094c15c5166f-kube-api-access-6bz8z\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.100570 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.100559    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/612f51d4-bab6-4b28-b837-094c15c5166f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.100829 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.100592    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/612f51d4-bab6-4b28-b837-094c15c5166f-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.100829 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.100684    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/612f51d4-bab6-4b28-b837-094c15c5166f-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.100829 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.100700    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/612f51d4-bab6-4b28-b837-094c15c5166f-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.101010 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.100985    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/612f51d4-bab6-4b28-b837-094c15c5166f-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.101137 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.101087    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/612f51d4-bab6-4b28-b837-094c15c5166f-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.101448 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.101425    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/612f51d4-bab6-4b28-b837-094c15c5166f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.103068 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.103042    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/612f51d4-bab6-4b28-b837-094c15c5166f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.103180 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.103053    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/612f51d4-bab6-4b28-b837-094c15c5166f-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.161636 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.161603    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/612f51d4-bab6-4b28-b837-094c15c5166f-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.161911 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.161887    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bz8z\" (UniqueName: \"kubernetes.io/projected/612f51d4-bab6-4b28-b837-094c15c5166f-kube-api-access-6bz8z\") pod \"maas-default-gateway-openshift-default-778b7b5c47-gnmf7\" (UID: \"612f51d4-bab6-4b28-b837-094c15c5166f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.426102 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.426023    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:50.867878 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.867845    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"]
Nov 02 02:09:50.869500 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:09:50.869474    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod612f51d4_bab6_4b28_b837_094c15c5166f.slice/crio-31e7c6d03f860e3dcc13da3a0fb27c061178ddfbf515b25dd4cab4b16919d317 WatchSource:0}: Error finding container 31e7c6d03f860e3dcc13da3a0fb27c061178ddfbf515b25dd4cab4b16919d317: Status 404 returned error can't find the container with id 31e7c6d03f860e3dcc13da3a0fb27c061178ddfbf515b25dd4cab4b16919d317
Nov 02 02:09:50.871913 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.871881    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:09:50.872009 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.871945    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:09:50.872009 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.871974    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:09:50.923240 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:50.923211    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7" event={"ID":"612f51d4-bab6-4b28-b837-094c15c5166f","Type":"ContainerStarted","Data":"31e7c6d03f860e3dcc13da3a0fb27c061178ddfbf515b25dd4cab4b16919d317"}
Nov 02 02:09:51.168608 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.168537    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"]
Nov 02 02:09:51.168767 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.168735    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk" podUID="b6442e7b-b2ad-404b-8f46-745b7c55c4ba" containerName="maas-api" containerID="cri-o://d8b9bff0e9bee21206b8d8bfb7a66051a9039c870a153592d1797db2add847d7" gracePeriod=30
Nov 02 02:09:51.509349 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.509324    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 02:09:51.520276 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.520238    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-metrics-certs\") pod \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\" (UID: \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\") "
Nov 02 02:09:51.520377 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.520293    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-maas-api-tls\") pod \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\" (UID: \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\") "
Nov 02 02:09:51.520448 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.520400    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-openshift-service-ca\") pod \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\" (UID: \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\") "
Nov 02 02:09:51.520577 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.520559    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-kube-api-access-pvjkh\" (UniqueName: \"kubernetes.io/projected/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-kube-api-access-pvjkh\") pod \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\" (UID: \"b6442e7b-b2ad-404b-8f46-745b7c55c4ba\") "
Nov 02 02:09:51.520991 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.520838    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-openshift-service-ca" pod "b6442e7b-b2ad-404b-8f46-745b7c55c4ba" (UID: "b6442e7b-b2ad-404b-8f46-745b7c55c4ba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:09:51.521104 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.521028    4515 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-openshift-service-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:51.522935 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.522909    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-metrics-certs" pod "b6442e7b-b2ad-404b-8f46-745b7c55c4ba" (UID: "b6442e7b-b2ad-404b-8f46-745b7c55c4ba"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:09:51.523316 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.523106    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-maas-api-tls" pod "b6442e7b-b2ad-404b-8f46-745b7c55c4ba" (UID: "b6442e7b-b2ad-404b-8f46-745b7c55c4ba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:09:51.523316 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.523284    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-kube-api-access-pvjkh" pod "b6442e7b-b2ad-404b-8f46-745b7c55c4ba" (UID: "b6442e7b-b2ad-404b-8f46-745b7c55c4ba"). InnerVolumeSpecName "kube-api-access-pvjkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:09:51.624014 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.623975    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvjkh\" (UniqueName: \"kubernetes.io/projected/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-kube-api-access-pvjkh\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:51.624014 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.624017    4515 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-metrics-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:51.624308 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.624030    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6442e7b-b2ad-404b-8f46-745b7c55c4ba-maas-api-tls\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:09:51.941215 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.937222    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7" event={"ID":"612f51d4-bab6-4b28-b837-094c15c5166f","Type":"ContainerStarted","Data":"7fb94869c5f72fc2190231b8dc74228c8d43bef1a065f550c8e927c95659820a"}
Nov 02 02:09:51.941215 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.939895    4515 generic.go:356] "Generic (PLEG): container finished" podID="b6442e7b-b2ad-404b-8f46-745b7c55c4ba" containerID="d8b9bff0e9bee21206b8d8bfb7a66051a9039c870a153592d1797db2add847d7" exitCode=0
Nov 02 02:09:51.941215 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.940059    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk" event={"ID":"b6442e7b-b2ad-404b-8f46-745b7c55c4ba","Type":"ContainerDied","Data":"d8b9bff0e9bee21206b8d8bfb7a66051a9039c870a153592d1797db2add847d7"}
Nov 02 02:09:51.941215 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.940103    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk" event={"ID":"b6442e7b-b2ad-404b-8f46-745b7c55c4ba","Type":"ContainerDied","Data":"21070ba3beeb924fa7cfb844fff4da4ac4f30d5bdc83a9d6d32a8a7ecaf3c204"}
Nov 02 02:09:51.941215 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.940160    4515 scope.go:122] "RemoveContainer" containerID="d8b9bff0e9bee21206b8d8bfb7a66051a9039c870a153592d1797db2add847d7"
Nov 02 02:09:51.941215 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.940307    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"
Nov 02 02:09:51.960437 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.960414    4515 scope.go:122] "RemoveContainer" containerID="d8b9bff0e9bee21206b8d8bfb7a66051a9039c870a153592d1797db2add847d7"
Nov 02 02:09:51.960708 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:09:51.960679    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8b9bff0e9bee21206b8d8bfb7a66051a9039c870a153592d1797db2add847d7\": container with ID starting with d8b9bff0e9bee21206b8d8bfb7a66051a9039c870a153592d1797db2add847d7 not found: ID does not exist" containerID="d8b9bff0e9bee21206b8d8bfb7a66051a9039c870a153592d1797db2add847d7"
Nov 02 02:09:51.960810 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:51.960714    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8b9bff0e9bee21206b8d8bfb7a66051a9039c870a153592d1797db2add847d7"} err="failed to get container status \"d8b9bff0e9bee21206b8d8bfb7a66051a9039c870a153592d1797db2add847d7\": rpc error: code = NotFound desc = could not find container \"d8b9bff0e9bee21206b8d8bfb7a66051a9039c870a153592d1797db2add847d7\": container with ID starting with d8b9bff0e9bee21206b8d8bfb7a66051a9039c870a153592d1797db2add847d7 not found: ID does not exist"
Nov 02 02:09:52.059107 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:52.059053    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7" podStartSLOduration=3.059040306 podStartE2EDuration="3.059040306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:09:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:09:52.057973466 +0000 UTC m=+2281.270343291" watchObservedRunningTime="2026-11-02 02:09:52.059040306 +0000 UTC m=+2281.271410114"
Nov 02 02:09:52.197762 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:52.190239    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"]
Nov 02 02:09:52.197762 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:52.192656    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-55bc84-56dbc9c8bf-jknbk"]
Nov 02 02:09:52.446154 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:52.444484    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:52.450187 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:52.449283    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-gnmf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 02:09:52.450187 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:52.449344    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7" podUID="612f51d4-bab6-4b28-b837-094c15c5166f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 02:09:52.955928 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:52.955891    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6442e7b-b2ad-404b-8f46-745b7c55c4ba" path="/var/lib/kubelet/pods/b6442e7b-b2ad-404b-8f46-745b7c55c4ba/volumes"
Nov 02 02:09:53.444146 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:53.441178    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-gnmf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 02:09:53.444146 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:53.441243    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7" podUID="612f51d4-bab6-4b28-b837-094c15c5166f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 02:09:54.451851 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:54.451816    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-gnmf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 02:09:54.454374 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:54.454330    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7" podUID="612f51d4-bab6-4b28-b837-094c15c5166f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 02:09:55.443697 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:55.443663    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-gnmf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 02:09:55.444486 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:55.444454    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7" podUID="612f51d4-bab6-4b28-b837-094c15c5166f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 02:09:56.446140 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:56.445219    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-gnmf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 02:09:56.446140 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:56.445303    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7" podUID="612f51d4-bab6-4b28-b837-094c15c5166f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 02:09:57.427133 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:57.427085    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-gnmf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 02:09:57.427312 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:57.427167    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7" podUID="612f51d4-bab6-4b28-b837-094c15c5166f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 02:09:58.427298 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:58.427262    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-gnmf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 02:09:58.427794 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:58.427320    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7" podUID="612f51d4-bab6-4b28-b837-094c15c5166f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 02:09:59.470303 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:59.470270    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:59.470816 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:59.470696    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:09:59.471918 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:09:59.471895    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7"
Nov 02 02:10:00.994038 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:00.994000    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"]
Nov 02 02:10:00.994564 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:00.994227    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d" podUID="bb1ff40c-c38c-474d-8522-6da91152739a" containerName="istio-proxy" containerID="cri-o://e82d1666e38b91caae756825a1e29bdca7a8a6a45c9907bd39b9bd6bdee23b6e" gracePeriod=30
Nov 02 02:10:02.987174 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:02.987079    4515 patch_prober.go:29] interesting pod/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 02:10:02.987557 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:02.987170    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d" podUID="bb1ff40c-c38c-474d-8522-6da91152739a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 02:10:06.259292 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.259264    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 02:10:06.359545 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.359464    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bb1ff40c-c38c-474d-8522-6da91152739a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb1ff40c-c38c-474d-8522-6da91152739a-istio-podinfo\") pod \"bb1ff40c-c38c-474d-8522-6da91152739a\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") "
Nov 02 02:10:06.359545 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.359514    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-credential-socket\") pod \"bb1ff40c-c38c-474d-8522-6da91152739a\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") "
Nov 02 02:10:06.359764 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.359563    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-istio-envoy\") pod \"bb1ff40c-c38c-474d-8522-6da91152739a\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") "
Nov 02 02:10:06.359764 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.359614    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bb1ff40c-c38c-474d-8522-6da91152739a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb1ff40c-c38c-474d-8522-6da91152739a-istiod-ca-cert\") pod \"bb1ff40c-c38c-474d-8522-6da91152739a\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") "
Nov 02 02:10:06.359764 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.359652    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb1ff40c-c38c-474d-8522-6da91152739a-istio-token\" (UniqueName: \"kubernetes.io/projected/bb1ff40c-c38c-474d-8522-6da91152739a-istio-token\") pod \"bb1ff40c-c38c-474d-8522-6da91152739a\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") "
Nov 02 02:10:06.359764 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.359693    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-workload-socket\") pod \"bb1ff40c-c38c-474d-8522-6da91152739a\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") "
Nov 02 02:10:06.359764 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.359724    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-istio-data\") pod \"bb1ff40c-c38c-474d-8522-6da91152739a\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") "
Nov 02 02:10:06.360019 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.359765    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb1ff40c-c38c-474d-8522-6da91152739a-kube-api-access-hdp8r\" (UniqueName: \"kubernetes.io/projected/bb1ff40c-c38c-474d-8522-6da91152739a-kube-api-access-hdp8r\") pod \"bb1ff40c-c38c-474d-8522-6da91152739a\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") "
Nov 02 02:10:06.360019 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.359879    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-credential-socket" pod "bb1ff40c-c38c-474d-8522-6da91152739a" (UID: "bb1ff40c-c38c-474d-8522-6da91152739a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:10:06.360019 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.359889    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-workload-certs\") pod \"bb1ff40c-c38c-474d-8522-6da91152739a\" (UID: \"bb1ff40c-c38c-474d-8522-6da91152739a\") "
Nov 02 02:10:06.360264 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.360108    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-workload-certs" pod "bb1ff40c-c38c-474d-8522-6da91152739a" (UID: "bb1ff40c-c38c-474d-8522-6da91152739a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:10:06.360264 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.360164    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-istio-data" pod "bb1ff40c-c38c-474d-8522-6da91152739a" (UID: "bb1ff40c-c38c-474d-8522-6da91152739a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:10:06.360264 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.360192    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb1ff40c-c38c-474d-8522-6da91152739a-istiod-ca-cert" pod "bb1ff40c-c38c-474d-8522-6da91152739a" (UID: "bb1ff40c-c38c-474d-8522-6da91152739a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:10:06.360386 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.360256    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-workload-socket" pod "bb1ff40c-c38c-474d-8522-6da91152739a" (UID: "bb1ff40c-c38c-474d-8522-6da91152739a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:10:06.360578 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.360556    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:06.360664 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.360584    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:06.360664 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.360599    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb1ff40c-c38c-474d-8522-6da91152739a-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:06.360664 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.360614    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:06.360664 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.360628    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:06.361808 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.361777    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bb1ff40c-c38c-474d-8522-6da91152739a-istio-podinfo" pod "bb1ff40c-c38c-474d-8522-6da91152739a" (UID: "bb1ff40c-c38c-474d-8522-6da91152739a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:10:06.361928 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.361892    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb1ff40c-c38c-474d-8522-6da91152739a-istio-token" pod "bb1ff40c-c38c-474d-8522-6da91152739a" (UID: "bb1ff40c-c38c-474d-8522-6da91152739a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:10:06.361982 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.361924    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-istio-envoy" pod "bb1ff40c-c38c-474d-8522-6da91152739a" (UID: "bb1ff40c-c38c-474d-8522-6da91152739a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:10:06.362291 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.362272    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb1ff40c-c38c-474d-8522-6da91152739a-kube-api-access-hdp8r" pod "bb1ff40c-c38c-474d-8522-6da91152739a" (UID: "bb1ff40c-c38c-474d-8522-6da91152739a"). InnerVolumeSpecName "kube-api-access-hdp8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:10:06.461648 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.461618    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb1ff40c-c38c-474d-8522-6da91152739a-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:06.461648 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.461649    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdp8r\" (UniqueName: \"kubernetes.io/projected/bb1ff40c-c38c-474d-8522-6da91152739a-kube-api-access-hdp8r\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:06.461837 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.461664    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb1ff40c-c38c-474d-8522-6da91152739a-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:06.461837 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:06.461674    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff40c-c38c-474d-8522-6da91152739a-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:07.041178 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:07.041144    4515 generic.go:356] "Generic (PLEG): container finished" podID="bb1ff40c-c38c-474d-8522-6da91152739a" containerID="e82d1666e38b91caae756825a1e29bdca7a8a6a45c9907bd39b9bd6bdee23b6e" exitCode=0
Nov 02 02:10:07.041373 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:07.041193    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"
Nov 02 02:10:07.041373 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:07.041229    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d" event={"ID":"bb1ff40c-c38c-474d-8522-6da91152739a","Type":"ContainerDied","Data":"e82d1666e38b91caae756825a1e29bdca7a8a6a45c9907bd39b9bd6bdee23b6e"}
Nov 02 02:10:07.041373 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:07.041302    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d" event={"ID":"bb1ff40c-c38c-474d-8522-6da91152739a","Type":"ContainerDied","Data":"ea69d45602e503e3b817d71925cb9b93477dd53c43687935ba9a5ed5af5c787a"}
Nov 02 02:10:07.041373 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:07.041330    4515 scope.go:122] "RemoveContainer" containerID="e82d1666e38b91caae756825a1e29bdca7a8a6a45c9907bd39b9bd6bdee23b6e"
Nov 02 02:10:07.050023 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:07.050007    4515 scope.go:122] "RemoveContainer" containerID="e82d1666e38b91caae756825a1e29bdca7a8a6a45c9907bd39b9bd6bdee23b6e"
Nov 02 02:10:07.050307 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:10:07.050287    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e82d1666e38b91caae756825a1e29bdca7a8a6a45c9907bd39b9bd6bdee23b6e\": container with ID starting with e82d1666e38b91caae756825a1e29bdca7a8a6a45c9907bd39b9bd6bdee23b6e not found: ID does not exist" containerID="e82d1666e38b91caae756825a1e29bdca7a8a6a45c9907bd39b9bd6bdee23b6e"
Nov 02 02:10:07.050356 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:07.050325    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e82d1666e38b91caae756825a1e29bdca7a8a6a45c9907bd39b9bd6bdee23b6e"} err="failed to get container status \"e82d1666e38b91caae756825a1e29bdca7a8a6a45c9907bd39b9bd6bdee23b6e\": rpc error: code = NotFound desc = could not find container \"e82d1666e38b91caae756825a1e29bdca7a8a6a45c9907bd39b9bd6bdee23b6e\": container with ID starting with e82d1666e38b91caae756825a1e29bdca7a8a6a45c9907bd39b9bd6bdee23b6e not found: ID does not exist"
Nov 02 02:10:07.255784 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:07.255702    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"]
Nov 02 02:10:07.305037 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:07.305007    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-55bc84-openshift-default-694b575d5c-9t87d"]
Nov 02 02:10:08.952926 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:08.952890    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bb1ff40c-c38c-474d-8522-6da91152739a" path="/var/lib/kubelet/pods/bb1ff40c-c38c-474d-8522-6da91152739a/volumes"
Nov 02 02:10:09.506032 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.505999    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"]
Nov 02 02:10:09.506444 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.506431    4515 cpu_manager.go:414] "containerMap: removing container" podUID="bb1ff40c-c38c-474d-8522-6da91152739a" containerName="istio-proxy"
Nov 02 02:10:09.506491 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.506445    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb1ff40c-c38c-474d-8522-6da91152739a" containerName="istio-proxy"
Nov 02 02:10:09.506491 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.506457    4515 cpu_manager.go:414] "containerMap: removing container" podUID="b6442e7b-b2ad-404b-8f46-745b7c55c4ba" containerName="maas-api"
Nov 02 02:10:09.506491 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.506462    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6442e7b-b2ad-404b-8f46-745b7c55c4ba" containerName="maas-api"
Nov 02 02:10:09.506918 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.506539    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="bb1ff40c-c38c-474d-8522-6da91152739a" containerName="istio-proxy"
Nov 02 02:10:09.506918 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.506548    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6442e7b-b2ad-404b-8f46-745b7c55c4ba" containerName="maas-api"
Nov 02 02:10:09.511019 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.510999    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.697737 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.697704    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53accdd7-36ef-4487-812b-b0715588261d-istio-token\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.697737 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.697737    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53accdd7-36ef-4487-812b-b0715588261d-istio-podinfo\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.698049 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.697828    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53accdd7-36ef-4487-812b-b0715588261d-istiod-ca-cert\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.698049 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.697876    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-workload-certs\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.698049 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.697967    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-credential-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.698242 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.698095    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmtgd\" (UniqueName: \"kubernetes.io/projected/53accdd7-36ef-4487-812b-b0715588261d-kube-api-access-rmtgd\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.698242 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.698177    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-istio-envoy\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.698353 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.698244    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-workload-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.698353 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.698271    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-istio-data\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.760063 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.759968    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"]
Nov 02 02:10:09.799937 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.799897    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53accdd7-36ef-4487-812b-b0715588261d-istiod-ca-cert\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.800152 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.799954    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-workload-certs\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.800152 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.800012    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-credential-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.800152 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.800082    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmtgd\" (UniqueName: \"kubernetes.io/projected/53accdd7-36ef-4487-812b-b0715588261d-kube-api-access-rmtgd\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.800152 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.800127    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-istio-envoy\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.800394 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.800166    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-workload-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.800394 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.800194    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-istio-data\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.800394 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.800308    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53accdd7-36ef-4487-812b-b0715588261d-istio-token\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.800394 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.800333    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53accdd7-36ef-4487-812b-b0715588261d-istio-podinfo\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.800648 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.800617    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-credential-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.800710 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.800654    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-istio-data\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.800985 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.800933    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-workload-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.801094 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.801018    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-workload-certs\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.801637 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.801607    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53accdd7-36ef-4487-812b-b0715588261d-istiod-ca-cert\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.803358 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.803333    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53accdd7-36ef-4487-812b-b0715588261d-istio-podinfo\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.803461 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.803358    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-istio-envoy\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.861627 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.861603    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53accdd7-36ef-4487-812b-b0715588261d-istio-token\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.861799 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:09.861781    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmtgd\" (UniqueName: \"kubernetes.io/projected/53accdd7-36ef-4487-812b-b0715588261d-kube-api-access-rmtgd\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:09.958875 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:10:09.958840    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:10:10.123979 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:10.123909    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:10.565454 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:10.565427    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"]
Nov 02 02:10:10.567468 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:10:10.567436    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53accdd7_36ef_4487_812b_b0715588261d.slice/crio-db4f43026b59c0bdfd898924b37ea22d374a0f72f28e8a84ef7b26be5e7c4bee WatchSource:0}: Error finding container db4f43026b59c0bdfd898924b37ea22d374a0f72f28e8a84ef7b26be5e7c4bee: Status 404 returned error can't find the container with id db4f43026b59c0bdfd898924b37ea22d374a0f72f28e8a84ef7b26be5e7c4bee
Nov 02 02:10:10.570166 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:10.570110    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:10:10.570272 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:10.570203    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:10:10.570272 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:10.570234    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:10:11.063190 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:11.063155    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9" event={"ID":"53accdd7-36ef-4487-812b-b0715588261d","Type":"ContainerStarted","Data":"ff98473e919f82fb47a99ef4323539d2a149b7bd2cc2e2a3193f2b10c391c126"}
Nov 02 02:10:11.063690 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:11.063210    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9" event={"ID":"53accdd7-36ef-4487-812b-b0715588261d","Type":"ContainerStarted","Data":"db4f43026b59c0bdfd898924b37ea22d374a0f72f28e8a84ef7b26be5e7c4bee"}
Nov 02 02:10:11.124445 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:11.124415    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:11.125907 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:11.125885    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 02:10:11.125975 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:11.125932    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9" podUID="53accdd7-36ef-4487-812b-b0715588261d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 02:10:11.284195 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:11.284137    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9" podStartSLOduration=2.284104976 podStartE2EDuration="2.284104976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:10:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:10:11.23399306 +0000 UTC m=+2300.446362868" watchObservedRunningTime="2026-11-02 02:10:11.284104976 +0000 UTC m=+2300.496474784"
Nov 02 02:10:12.146151 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:12.140552    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 02:10:12.146151 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:12.140621    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9" podUID="53accdd7-36ef-4487-812b-b0715588261d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 02:10:13.139160 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:13.138677    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 02:10:13.139160 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:13.138868    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9" podUID="53accdd7-36ef-4487-812b-b0715588261d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 02:10:14.145142 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:14.143536    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 02:10:14.145142 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:14.143612    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9" podUID="53accdd7-36ef-4487-812b-b0715588261d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 02:10:15.139711 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:15.139268    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 02:10:15.139711 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:15.139331    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9" podUID="53accdd7-36ef-4487-812b-b0715588261d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 02:10:16.140143 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:16.136474    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 02:10:16.140143 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:16.136537    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9" podUID="53accdd7-36ef-4487-812b-b0715588261d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 02:10:17.215481 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:17.215447    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:17.215967 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:17.215790    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:17.216767 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:17.216747    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:10:24.613126 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.613076    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"]
Nov 02 02:10:24.616881 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.616859    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.720583 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.720547    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"]
Nov 02 02:10:24.744464 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.744431    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-envoy\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.744674 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.744491    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-data\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.744674 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.744631    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-workload-certs\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.744785 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.744676    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxx2k\" (UniqueName: \"kubernetes.io/projected/d05354b1-b462-4af4-af05-80c5a8c0de70-kube-api-access-xxx2k\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.744835 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.744789    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-token\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.744835 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.744824    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-credential-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.744934 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.744919    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-workload-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.744980 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.744947    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d05354b1-b462-4af4-af05-80c5a8c0de70-istiod-ca-cert\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.745025 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.745014    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-podinfo\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.845890 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.845855    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-workload-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.845890 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.845896    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d05354b1-b462-4af4-af05-80c5a8c0de70-istiod-ca-cert\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.846144 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.845950    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-podinfo\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.846144 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.845980    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-envoy\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.846144 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.846017    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-data\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.846144 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.846058    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-workload-certs\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.846144 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.846074    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxx2k\" (UniqueName: \"kubernetes.io/projected/d05354b1-b462-4af4-af05-80c5a8c0de70-kube-api-access-xxx2k\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.846144 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.846107    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-token\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.846144 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.846146    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-credential-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.846547 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.846373    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-workload-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.846547 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.846459    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-data\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.846725 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.846703    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-credential-socket\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.846772 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.846734    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-workload-certs\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.846772 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.846746    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d05354b1-b462-4af4-af05-80c5a8c0de70-istiod-ca-cert\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.848394 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.848371    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-envoy\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.848535 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.848516    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-podinfo\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.910262 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.910183    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-token\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.910424 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.910388    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxx2k\" (UniqueName: \"kubernetes.io/projected/d05354b1-b462-4af4-af05-80c5a8c0de70-kube-api-access-xxx2k\") pod \"e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") " pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:24.930298 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:24.930270    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:25.360640 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:25.360485    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"]
Nov 02 02:10:25.363307 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:10:25.363269    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd05354b1_b462_4af4_af05_80c5a8c0de70.slice/crio-dba42a1c2fb89880d04127094dda11773a020cdd14d537e273a0f949483a36ed WatchSource:0}: Error finding container dba42a1c2fb89880d04127094dda11773a020cdd14d537e273a0f949483a36ed: Status 404 returned error can't find the container with id dba42a1c2fb89880d04127094dda11773a020cdd14d537e273a0f949483a36ed
Nov 02 02:10:25.365686 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:25.365649    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:10:25.365760 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:25.365717    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:10:25.365760 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:25.365746    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:10:26.154460 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:26.154424    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml" event={"ID":"d05354b1-b462-4af4-af05-80c5a8c0de70","Type":"ContainerStarted","Data":"7d4151a470ca832294053335e13a694e199b838c901c54252003cafa195de754"}
Nov 02 02:10:26.154975 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:26.154488    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml" event={"ID":"d05354b1-b462-4af4-af05-80c5a8c0de70","Type":"ContainerStarted","Data":"dba42a1c2fb89880d04127094dda11773a020cdd14d537e273a0f949483a36ed"}
Nov 02 02:10:26.277084 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:26.277019    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml" podStartSLOduration=2.276997898 podStartE2EDuration="2.276997898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:10:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:10:26.274477144 +0000 UTC m=+2315.486846953" watchObservedRunningTime="2026-11-02 02:10:26.276997898 +0000 UTC m=+2315.489367706"
Nov 02 02:10:26.938550 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:26.938514    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:26.945284 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:26.945245    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 02:10:26.945529 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:26.945500    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml" podUID="d05354b1-b462-4af4-af05-80c5a8c0de70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 02:10:27.949250 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:27.949204    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 02:10:27.949667 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:27.949284    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml" podUID="d05354b1-b462-4af4-af05-80c5a8c0de70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 02:10:28.264896 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:28.264806    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"]
Nov 02 02:10:28.265322 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:28.265293    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8" podUID="be97e917-9b14-484e-90f1-b9ad1b678313" containerName="istio-proxy" containerID="cri-o://168bd5a7c541f48cbc9c03191de137bff160a68be9ccf1aa0d5bcbdf2480f1a1" gracePeriod=30
Nov 02 02:10:28.937579 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:28.937535    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 02:10:28.937915 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:28.937886    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml" podUID="d05354b1-b462-4af4-af05-80c5a8c0de70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 02:10:29.939550 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:29.939510    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 02:10:29.940232 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:29.940194    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml" podUID="d05354b1-b462-4af4-af05-80c5a8c0de70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 02:10:30.936595 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:30.936556    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 02:10:30.936891 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:30.936858    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml" podUID="d05354b1-b462-4af4-af05-80c5a8c0de70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 02:10:31.935759 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:31.935731    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:31.936174 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:31.936156    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:31.936938 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:31.936922    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:10:32.059637 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:32.059600    4515 patch_prober.go:29] interesting pod/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 02:10:32.059797 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:32.059656    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8" podUID="be97e917-9b14-484e-90f1-b9ad1b678313" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 02:10:32.953633 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:32.953602    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"]
Nov 02 02:10:32.954040 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:32.953795    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2" podUID="b05cc018-433c-4f3a-9708-95e3bdde50ff" containerName="istio-proxy" containerID="cri-o://c826576cbc49668c03a27ae04712cffdcd28a52c74ea8affcfd33a1793e62c35" gracePeriod=30
Nov 02 02:10:33.561797 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.561760    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 02:10:33.603178 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.603143    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be97e917-9b14-484e-90f1-b9ad1b678313-istio-token\" (UniqueName: \"kubernetes.io/projected/be97e917-9b14-484e-90f1-b9ad1b678313-istio-token\") pod \"be97e917-9b14-484e-90f1-b9ad1b678313\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") "
Nov 02 02:10:33.603381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.603194    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-credential-socket\") pod \"be97e917-9b14-484e-90f1-b9ad1b678313\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") "
Nov 02 02:10:33.603381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.603238    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/be97e917-9b14-484e-90f1-b9ad1b678313-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be97e917-9b14-484e-90f1-b9ad1b678313-istio-podinfo\") pod \"be97e917-9b14-484e-90f1-b9ad1b678313\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") "
Nov 02 02:10:33.603381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.603303    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be97e917-9b14-484e-90f1-b9ad1b678313-kube-api-access-n52w2\" (UniqueName: \"kubernetes.io/projected/be97e917-9b14-484e-90f1-b9ad1b678313-kube-api-access-n52w2\") pod \"be97e917-9b14-484e-90f1-b9ad1b678313\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") "
Nov 02 02:10:33.603381 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.603334    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-istio-data\") pod \"be97e917-9b14-484e-90f1-b9ad1b678313\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") "
Nov 02 02:10:33.603600 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.603429    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-workload-socket\") pod \"be97e917-9b14-484e-90f1-b9ad1b678313\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") "
Nov 02 02:10:33.603600 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.603503    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be97e917-9b14-484e-90f1-b9ad1b678313-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be97e917-9b14-484e-90f1-b9ad1b678313-istiod-ca-cert\") pod \"be97e917-9b14-484e-90f1-b9ad1b678313\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") "
Nov 02 02:10:33.603600 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.603568    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-workload-certs\") pod \"be97e917-9b14-484e-90f1-b9ad1b678313\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") "
Nov 02 02:10:33.603746 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.603666    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-istio-envoy\") pod \"be97e917-9b14-484e-90f1-b9ad1b678313\" (UID: \"be97e917-9b14-484e-90f1-b9ad1b678313\") "
Nov 02 02:10:33.603967 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.603563    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-credential-socket" pod "be97e917-9b14-484e-90f1-b9ad1b678313" (UID: "be97e917-9b14-484e-90f1-b9ad1b678313"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:10:33.604100 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.604082    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:33.604100 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.604085    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be97e917-9b14-484e-90f1-b9ad1b678313-istiod-ca-cert" pod "be97e917-9b14-484e-90f1-b9ad1b678313" (UID: "be97e917-9b14-484e-90f1-b9ad1b678313"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:10:33.604264 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.603725    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-istio-data" pod "be97e917-9b14-484e-90f1-b9ad1b678313" (UID: "be97e917-9b14-484e-90f1-b9ad1b678313"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:10:33.604264 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.603760    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-workload-socket" pod "be97e917-9b14-484e-90f1-b9ad1b678313" (UID: "be97e917-9b14-484e-90f1-b9ad1b678313"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:10:33.604264 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.603918    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-workload-certs" pod "be97e917-9b14-484e-90f1-b9ad1b678313" (UID: "be97e917-9b14-484e-90f1-b9ad1b678313"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:10:33.606034 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.605998    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/be97e917-9b14-484e-90f1-b9ad1b678313-istio-podinfo" pod "be97e917-9b14-484e-90f1-b9ad1b678313" (UID: "be97e917-9b14-484e-90f1-b9ad1b678313"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:10:33.606146 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.606011    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be97e917-9b14-484e-90f1-b9ad1b678313-istio-token" pod "be97e917-9b14-484e-90f1-b9ad1b678313" (UID: "be97e917-9b14-484e-90f1-b9ad1b678313"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:10:33.606146 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.606035    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be97e917-9b14-484e-90f1-b9ad1b678313-kube-api-access-n52w2" pod "be97e917-9b14-484e-90f1-b9ad1b678313" (UID: "be97e917-9b14-484e-90f1-b9ad1b678313"). InnerVolumeSpecName "kube-api-access-n52w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:10:33.606265 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.606246    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-istio-envoy" pod "be97e917-9b14-484e-90f1-b9ad1b678313" (UID: "be97e917-9b14-484e-90f1-b9ad1b678313"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:10:33.705791 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.705749    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:33.705791 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.705788    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:33.706033 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.705802    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be97e917-9b14-484e-90f1-b9ad1b678313-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:33.706033 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.705816    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be97e917-9b14-484e-90f1-b9ad1b678313-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:33.706033 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.705829    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n52w2\" (UniqueName: \"kubernetes.io/projected/be97e917-9b14-484e-90f1-b9ad1b678313-kube-api-access-n52w2\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:33.706033 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.705843    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:33.706033 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.705855    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be97e917-9b14-484e-90f1-b9ad1b678313-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:33.706033 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:33.705868    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be97e917-9b14-484e-90f1-b9ad1b678313-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:34.210010 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:34.209972    4515 generic.go:356] "Generic (PLEG): container finished" podID="be97e917-9b14-484e-90f1-b9ad1b678313" containerID="168bd5a7c541f48cbc9c03191de137bff160a68be9ccf1aa0d5bcbdf2480f1a1" exitCode=0
Nov 02 02:10:34.210529 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:34.210027    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"
Nov 02 02:10:34.210529 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:34.210060    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8" event={"ID":"be97e917-9b14-484e-90f1-b9ad1b678313","Type":"ContainerDied","Data":"168bd5a7c541f48cbc9c03191de137bff160a68be9ccf1aa0d5bcbdf2480f1a1"}
Nov 02 02:10:34.210529 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:34.210149    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8" event={"ID":"be97e917-9b14-484e-90f1-b9ad1b678313","Type":"ContainerDied","Data":"fa77b55f479773fe99a2e59b9d946f54e5664a9e4f69ccaa989c6325cb6b1f4d"}
Nov 02 02:10:34.210529 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:34.210187    4515 scope.go:122] "RemoveContainer" containerID="168bd5a7c541f48cbc9c03191de137bff160a68be9ccf1aa0d5bcbdf2480f1a1"
Nov 02 02:10:34.220070 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:34.220045    4515 scope.go:122] "RemoveContainer" containerID="168bd5a7c541f48cbc9c03191de137bff160a68be9ccf1aa0d5bcbdf2480f1a1"
Nov 02 02:10:34.220427 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:10:34.220404    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"168bd5a7c541f48cbc9c03191de137bff160a68be9ccf1aa0d5bcbdf2480f1a1\": container with ID starting with 168bd5a7c541f48cbc9c03191de137bff160a68be9ccf1aa0d5bcbdf2480f1a1 not found: ID does not exist" containerID="168bd5a7c541f48cbc9c03191de137bff160a68be9ccf1aa0d5bcbdf2480f1a1"
Nov 02 02:10:34.220485 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:34.220437    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"168bd5a7c541f48cbc9c03191de137bff160a68be9ccf1aa0d5bcbdf2480f1a1"} err="failed to get container status \"168bd5a7c541f48cbc9c03191de137bff160a68be9ccf1aa0d5bcbdf2480f1a1\": rpc error: code = NotFound desc = could not find container \"168bd5a7c541f48cbc9c03191de137bff160a68be9ccf1aa0d5bcbdf2480f1a1\": container with ID starting with 168bd5a7c541f48cbc9c03191de137bff160a68be9ccf1aa0d5bcbdf2480f1a1 not found: ID does not exist"
Nov 02 02:10:34.376814 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:34.376777    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"]
Nov 02 02:10:34.426246 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:34.426212    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-061fab-openshift-default-d6b5b6c67-c26f8"]
Nov 02 02:10:34.952712 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:34.952678    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be97e917-9b14-484e-90f1-b9ad1b678313" path="/var/lib/kubelet/pods/be97e917-9b14-484e-90f1-b9ad1b678313/volumes"
Nov 02 02:10:38.207873 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.207846    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 02:10:38.234597 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.234512    4515 generic.go:356] "Generic (PLEG): container finished" podID="b05cc018-433c-4f3a-9708-95e3bdde50ff" containerID="c826576cbc49668c03a27ae04712cffdcd28a52c74ea8affcfd33a1793e62c35" exitCode=0
Nov 02 02:10:38.234597 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.234567    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"
Nov 02 02:10:38.234597 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.234582    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2" event={"ID":"b05cc018-433c-4f3a-9708-95e3bdde50ff","Type":"ContainerDied","Data":"c826576cbc49668c03a27ae04712cffdcd28a52c74ea8affcfd33a1793e62c35"}
Nov 02 02:10:38.234820 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.234664    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2" event={"ID":"b05cc018-433c-4f3a-9708-95e3bdde50ff","Type":"ContainerDied","Data":"e26b80e055255eced077a109b6957e34c5cd89c4718b03e3a073c48d00e3c33f"}
Nov 02 02:10:38.234820 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.234695    4515 scope.go:122] "RemoveContainer" containerID="c826576cbc49668c03a27ae04712cffdcd28a52c74ea8affcfd33a1793e62c35"
Nov 02 02:10:38.243546 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.243526    4515 scope.go:122] "RemoveContainer" containerID="c826576cbc49668c03a27ae04712cffdcd28a52c74ea8affcfd33a1793e62c35"
Nov 02 02:10:38.243849 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:10:38.243823    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c826576cbc49668c03a27ae04712cffdcd28a52c74ea8affcfd33a1793e62c35\": container with ID starting with c826576cbc49668c03a27ae04712cffdcd28a52c74ea8affcfd33a1793e62c35 not found: ID does not exist" containerID="c826576cbc49668c03a27ae04712cffdcd28a52c74ea8affcfd33a1793e62c35"
Nov 02 02:10:38.243919 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.243860    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c826576cbc49668c03a27ae04712cffdcd28a52c74ea8affcfd33a1793e62c35"} err="failed to get container status \"c826576cbc49668c03a27ae04712cffdcd28a52c74ea8affcfd33a1793e62c35\": rpc error: code = NotFound desc = could not find container \"c826576cbc49668c03a27ae04712cffdcd28a52c74ea8affcfd33a1793e62c35\": container with ID starting with c826576cbc49668c03a27ae04712cffdcd28a52c74ea8affcfd33a1793e62c35 not found: ID does not exist"
Nov 02 02:10:38.259959 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.259932    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-data\") pod \"b05cc018-433c-4f3a-9708-95e3bdde50ff\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") "
Nov 02 02:10:38.260105 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.259969    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-token\" (UniqueName: \"kubernetes.io/projected/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-token\") pod \"b05cc018-433c-4f3a-9708-95e3bdde50ff\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") "
Nov 02 02:10:38.260105 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.259997    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-podinfo\") pod \"b05cc018-433c-4f3a-9708-95e3bdde50ff\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") "
Nov 02 02:10:38.260105 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.260035    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-envoy\") pod \"b05cc018-433c-4f3a-9708-95e3bdde50ff\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") "
Nov 02 02:10:38.260105 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.260079    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-credential-socket\") pod \"b05cc018-433c-4f3a-9708-95e3bdde50ff\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") "
Nov 02 02:10:38.260372 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.260111    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-workload-certs\") pod \"b05cc018-433c-4f3a-9708-95e3bdde50ff\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") "
Nov 02 02:10:38.260372 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.260163    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b05cc018-433c-4f3a-9708-95e3bdde50ff-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b05cc018-433c-4f3a-9708-95e3bdde50ff-istiod-ca-cert\") pod \"b05cc018-433c-4f3a-9708-95e3bdde50ff\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") "
Nov 02 02:10:38.260372 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.260280    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-workload-socket\") pod \"b05cc018-433c-4f3a-9708-95e3bdde50ff\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") "
Nov 02 02:10:38.260372 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.260313    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b05cc018-433c-4f3a-9708-95e3bdde50ff-kube-api-access-b7qvd\" (UniqueName: \"kubernetes.io/projected/b05cc018-433c-4f3a-9708-95e3bdde50ff-kube-api-access-b7qvd\") pod \"b05cc018-433c-4f3a-9708-95e3bdde50ff\" (UID: \"b05cc018-433c-4f3a-9708-95e3bdde50ff\") "
Nov 02 02:10:38.260372 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.260324    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-data" pod "b05cc018-433c-4f3a-9708-95e3bdde50ff" (UID: "b05cc018-433c-4f3a-9708-95e3bdde50ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:10:38.260626 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.260498    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-credential-socket" pod "b05cc018-433c-4f3a-9708-95e3bdde50ff" (UID: "b05cc018-433c-4f3a-9708-95e3bdde50ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:10:38.260687 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.260632    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-workload-certs" pod "b05cc018-433c-4f3a-9708-95e3bdde50ff" (UID: "b05cc018-433c-4f3a-9708-95e3bdde50ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:10:38.260687 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.260654    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b05cc018-433c-4f3a-9708-95e3bdde50ff-istiod-ca-cert" pod "b05cc018-433c-4f3a-9708-95e3bdde50ff" (UID: "b05cc018-433c-4f3a-9708-95e3bdde50ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:10:38.260832 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.260813    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-workload-socket" pod "b05cc018-433c-4f3a-9708-95e3bdde50ff" (UID: "b05cc018-433c-4f3a-9708-95e3bdde50ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:10:38.260930 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.260910    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:38.260963 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.260940    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:38.260963 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.260958    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:38.261023 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.260972    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b05cc018-433c-4f3a-9708-95e3bdde50ff-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:38.262302 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.262282    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-token" pod "b05cc018-433c-4f3a-9708-95e3bdde50ff" (UID: "b05cc018-433c-4f3a-9708-95e3bdde50ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:10:38.262448 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.262433    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b05cc018-433c-4f3a-9708-95e3bdde50ff-kube-api-access-b7qvd" pod "b05cc018-433c-4f3a-9708-95e3bdde50ff" (UID: "b05cc018-433c-4f3a-9708-95e3bdde50ff"). InnerVolumeSpecName "kube-api-access-b7qvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:10:38.262707 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.262687    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-podinfo" pod "b05cc018-433c-4f3a-9708-95e3bdde50ff" (UID: "b05cc018-433c-4f3a-9708-95e3bdde50ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:10:38.262767 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.262685    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-envoy" pod "b05cc018-433c-4f3a-9708-95e3bdde50ff" (UID: "b05cc018-433c-4f3a-9708-95e3bdde50ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:10:38.362516 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.362481    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:38.362516 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.362510    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:38.362516 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.362521    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:38.362751 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.362530    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05cc018-433c-4f3a-9708-95e3bdde50ff-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:38.362751 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.362539    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7qvd\" (UniqueName: \"kubernetes.io/projected/b05cc018-433c-4f3a-9708-95e3bdde50ff-kube-api-access-b7qvd\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:10:38.656040 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.656008    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"]
Nov 02 02:10:38.660322 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.660293    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-qlft2"]
Nov 02 02:10:38.953683 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:10:38.953649    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b05cc018-433c-4f3a-9708-95e3bdde50ff" path="/var/lib/kubelet/pods/b05cc018-433c-4f3a-9708-95e3bdde50ff/volumes"
Nov 02 02:10:39.958990 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:10:39.958954    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:11:09.959094 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:11:09.959054    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:11:27.138506 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:27.138468    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"]
Nov 02 02:11:27.139050 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:27.138745    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml" podUID="d05354b1-b462-4af4-af05-80c5a8c0de70" containerName="istio-proxy" containerID="cri-o://7d4151a470ca832294053335e13a694e199b838c901c54252003cafa195de754" gracePeriod=30
Nov 02 02:11:27.151633 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:27.151608    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"]
Nov 02 02:11:27.151834 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:27.151814    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp" podUID="c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" containerName="istio-proxy" containerID="cri-o://0d7e71131516091164d9a188442bd4851c7653b291cc67530ccedf88602e3044" gracePeriod=30
Nov 02 02:11:27.173805 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:27.173777    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"]
Nov 02 02:11:27.173964 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:27.173946    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9" podUID="53accdd7-36ef-4487-812b-b0715588261d" containerName="istio-proxy" containerID="cri-o://ff98473e919f82fb47a99ef4323539d2a149b7bd2cc2e2a3193f2b10c391c126" gracePeriod=30
Nov 02 02:11:31.939009 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:31.938969    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 02:11:31.939397 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:31.939028    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml" podUID="d05354b1-b462-4af4-af05-80c5a8c0de70" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 02:11:32.217112 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.217088    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 02:11:32.217221 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.217154    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9" podUID="53accdd7-36ef-4487-812b-b0715588261d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 02:11:32.291900 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.291870    4515 patch_prober.go:29] interesting pod/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 02 02:11:32.292029 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.291925    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp" podUID="c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 02 02:11:32.492438 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.492403    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:11:32.521147 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.521091    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:11:32.532584 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.532563    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:11:32.579558 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.579525    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"
Nov 02 02:11:32.580168 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.580030    4515 generic.go:356] "Generic (PLEG): container finished" podID="d05354b1-b462-4af4-af05-80c5a8c0de70" containerID="7d4151a470ca832294053335e13a694e199b838c901c54252003cafa195de754" exitCode=0
Nov 02 02:11:32.580168 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.580197    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml" event={"ID":"d05354b1-b462-4af4-af05-80c5a8c0de70","Type":"ContainerDied","Data":"7d4151a470ca832294053335e13a694e199b838c901c54252003cafa195de754"}
Nov 02 02:11:32.580168 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.580250    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml" event={"ID":"d05354b1-b462-4af4-af05-80c5a8c0de70","Type":"ContainerDied","Data":"dba42a1c2fb89880d04127094dda11773a020cdd14d537e273a0f949483a36ed"}
Nov 02 02:11:32.580168 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.580285    4515 scope.go:122] "RemoveContainer" containerID="7d4151a470ca832294053335e13a694e199b838c901c54252003cafa195de754"
Nov 02 02:11:32.582838 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.582818    4515 generic.go:356] "Generic (PLEG): container finished" podID="c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" containerID="0d7e71131516091164d9a188442bd4851c7653b291cc67530ccedf88602e3044" exitCode=0
Nov 02 02:11:32.582937 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.582890    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"
Nov 02 02:11:32.583011 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.582986    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp" event={"ID":"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc","Type":"ContainerDied","Data":"0d7e71131516091164d9a188442bd4851c7653b291cc67530ccedf88602e3044"}
Nov 02 02:11:32.583096 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.583040    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp" event={"ID":"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc","Type":"ContainerDied","Data":"eca9b584f6f7248a43f8823d371cf16bb60d95195113d71699b5ea2fc26d3dd4"}
Nov 02 02:11:32.585943 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.585921    4515 generic.go:356] "Generic (PLEG): container finished" podID="53accdd7-36ef-4487-812b-b0715588261d" containerID="ff98473e919f82fb47a99ef4323539d2a149b7bd2cc2e2a3193f2b10c391c126" exitCode=0
Nov 02 02:11:32.586060 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.585971    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9" event={"ID":"53accdd7-36ef-4487-812b-b0715588261d","Type":"ContainerDied","Data":"ff98473e919f82fb47a99ef4323539d2a149b7bd2cc2e2a3193f2b10c391c126"}
Nov 02 02:11:32.586060 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.586014    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9" event={"ID":"53accdd7-36ef-4487-812b-b0715588261d","Type":"ContainerDied","Data":"db4f43026b59c0bdfd898924b37ea22d374a0f72f28e8a84ef7b26be5e7c4bee"}
Nov 02 02:11:32.586325 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.586083    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"
Nov 02 02:11:32.594024 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.594002    4515 scope.go:122] "RemoveContainer" containerID="7d4151a470ca832294053335e13a694e199b838c901c54252003cafa195de754"
Nov 02 02:11:32.594340 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:11:32.594316    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d4151a470ca832294053335e13a694e199b838c901c54252003cafa195de754\": container with ID starting with 7d4151a470ca832294053335e13a694e199b838c901c54252003cafa195de754 not found: ID does not exist" containerID="7d4151a470ca832294053335e13a694e199b838c901c54252003cafa195de754"
Nov 02 02:11:32.594415 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.594351    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d4151a470ca832294053335e13a694e199b838c901c54252003cafa195de754"} err="failed to get container status \"7d4151a470ca832294053335e13a694e199b838c901c54252003cafa195de754\": rpc error: code = NotFound desc = could not find container \"7d4151a470ca832294053335e13a694e199b838c901c54252003cafa195de754\": container with ID starting with 7d4151a470ca832294053335e13a694e199b838c901c54252003cafa195de754 not found: ID does not exist"
Nov 02 02:11:32.594415 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.594369    4515 scope.go:122] "RemoveContainer" containerID="0d7e71131516091164d9a188442bd4851c7653b291cc67530ccedf88602e3044"
Nov 02 02:11:32.605129 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.605064    4515 scope.go:122] "RemoveContainer" containerID="0d7e71131516091164d9a188442bd4851c7653b291cc67530ccedf88602e3044"
Nov 02 02:11:32.605818 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:11:32.605794    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d7e71131516091164d9a188442bd4851c7653b291cc67530ccedf88602e3044\": container with ID starting with 0d7e71131516091164d9a188442bd4851c7653b291cc67530ccedf88602e3044 not found: ID does not exist" containerID="0d7e71131516091164d9a188442bd4851c7653b291cc67530ccedf88602e3044"
Nov 02 02:11:32.605896 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.605824    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d7e71131516091164d9a188442bd4851c7653b291cc67530ccedf88602e3044"} err="failed to get container status \"0d7e71131516091164d9a188442bd4851c7653b291cc67530ccedf88602e3044\": rpc error: code = NotFound desc = could not find container \"0d7e71131516091164d9a188442bd4851c7653b291cc67530ccedf88602e3044\": container with ID starting with 0d7e71131516091164d9a188442bd4851c7653b291cc67530ccedf88602e3044 not found: ID does not exist"
Nov 02 02:11:32.605896 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.605843    4515 scope.go:122] "RemoveContainer" containerID="ff98473e919f82fb47a99ef4323539d2a149b7bd2cc2e2a3193f2b10c391c126"
Nov 02 02:11:32.614721 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.614702    4515 scope.go:122] "RemoveContainer" containerID="ff98473e919f82fb47a99ef4323539d2a149b7bd2cc2e2a3193f2b10c391c126"
Nov 02 02:11:32.615023 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:11:32.615003    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff98473e919f82fb47a99ef4323539d2a149b7bd2cc2e2a3193f2b10c391c126\": container with ID starting with ff98473e919f82fb47a99ef4323539d2a149b7bd2cc2e2a3193f2b10c391c126 not found: ID does not exist" containerID="ff98473e919f82fb47a99ef4323539d2a149b7bd2cc2e2a3193f2b10c391c126"
Nov 02 02:11:32.615079 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.615034    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff98473e919f82fb47a99ef4323539d2a149b7bd2cc2e2a3193f2b10c391c126"} err="failed to get container status \"ff98473e919f82fb47a99ef4323539d2a149b7bd2cc2e2a3193f2b10c391c126\": rpc error: code = NotFound desc = could not find container \"ff98473e919f82fb47a99ef4323539d2a149b7bd2cc2e2a3193f2b10c391c126\": container with ID starting with ff98473e919f82fb47a99ef4323539d2a149b7bd2cc2e2a3193f2b10c391c126 not found: ID does not exist"
Nov 02 02:11:32.677848 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.677815    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-credential-socket\") pod \"d05354b1-b462-4af4-af05-80c5a8c0de70\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") "
Nov 02 02:11:32.678170 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.677947    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-envoy\") pod \"d05354b1-b462-4af4-af05-80c5a8c0de70\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") "
Nov 02 02:11:32.678170 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678004    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-podinfo\") pod \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") "
Nov 02 02:11:32.678170 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678054    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-workload-certs\") pod \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") "
Nov 02 02:11:32.678170 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678087    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-credential-socket" pod "d05354b1-b462-4af4-af05-80c5a8c0de70" (UID: "d05354b1-b462-4af4-af05-80c5a8c0de70"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:11:32.678170 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678093    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-workload-socket\") pod \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678173    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-token\" (UniqueName: \"kubernetes.io/projected/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-token\") pod \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678217    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-istio-envoy\") pod \"53accdd7-36ef-4487-812b-b0715588261d\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678243    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-podinfo\") pod \"d05354b1-b462-4af4-af05-80c5a8c0de70\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678271    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istiod-ca-cert\") pod \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678335    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-workload-certs\") pod \"53accdd7-36ef-4487-812b-b0715588261d\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678350    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-workload-socket" pod "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" (UID: "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678372    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53accdd7-36ef-4487-812b-b0715588261d-istio-token\" (UniqueName: \"kubernetes.io/projected/53accdd7-36ef-4487-812b-b0715588261d-istio-token\") pod \"53accdd7-36ef-4487-812b-b0715588261d\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678380    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-workload-certs" pod "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" (UID: "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678404    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-workload-socket\") pod \"53accdd7-36ef-4487-812b-b0715588261d\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678433    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-token\" (UniqueName: \"kubernetes.io/projected/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-token\") pod \"d05354b1-b462-4af4-af05-80c5a8c0de70\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678468    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-credential-socket\") pod \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678494    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-istio-data\") pod \"53accdd7-36ef-4487-812b-b0715588261d\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678522    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-envoy\") pod \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678600    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-workload-certs" pod "53accdd7-36ef-4487-812b-b0715588261d" (UID: "53accdd7-36ef-4487-812b-b0715588261d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678640    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53accdd7-36ef-4487-812b-b0715588261d-kube-api-access-rmtgd\" (UniqueName: \"kubernetes.io/projected/53accdd7-36ef-4487-812b-b0715588261d-kube-api-access-rmtgd\") pod \"53accdd7-36ef-4487-812b-b0715588261d\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678668    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istiod-ca-cert" pod "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" (UID: "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678721    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/53accdd7-36ef-4487-812b-b0715588261d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53accdd7-36ef-4487-812b-b0715588261d-istiod-ca-cert\") pod \"53accdd7-36ef-4487-812b-b0715588261d\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678758    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-data\") pod \"d05354b1-b462-4af4-af05-80c5a8c0de70\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678787    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-kube-api-access-jkxcx\" (UniqueName: \"kubernetes.io/projected/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-kube-api-access-jkxcx\") pod \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678817    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-data\") pod \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\" (UID: \"c70d92a5-6ed0-470d-9c5d-822ffc97bbbc\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678851    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-credential-socket\") pod \"53accdd7-36ef-4487-812b-b0715588261d\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678903    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d05354b1-b462-4af4-af05-80c5a8c0de70-kube-api-access-xxx2k\" (UniqueName: \"kubernetes.io/projected/d05354b1-b462-4af4-af05-80c5a8c0de70-kube-api-access-xxx2k\") pod \"d05354b1-b462-4af4-af05-80c5a8c0de70\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.678975    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d05354b1-b462-4af4-af05-80c5a8c0de70-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d05354b1-b462-4af4-af05-80c5a8c0de70-istiod-ca-cert\") pod \"d05354b1-b462-4af4-af05-80c5a8c0de70\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") "
Nov 02 02:11:32.678988 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.679008    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-workload-certs\") pod \"d05354b1-b462-4af4-af05-80c5a8c0de70\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") "
Nov 02 02:11:32.681051 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.679052    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-workload-socket\") pod \"d05354b1-b462-4af4-af05-80c5a8c0de70\" (UID: \"d05354b1-b462-4af4-af05-80c5a8c0de70\") "
Nov 02 02:11:32.681051 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.679148    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/53accdd7-36ef-4487-812b-b0715588261d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53accdd7-36ef-4487-812b-b0715588261d-istio-podinfo\") pod \"53accdd7-36ef-4487-812b-b0715588261d\" (UID: \"53accdd7-36ef-4487-812b-b0715588261d\") "
Nov 02 02:11:32.681051 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.679682    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.681051 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.679704    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.681051 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.679721    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.681051 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.679735    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.681051 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.679751    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.681051 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.680443    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-data" pod "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" (UID: "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:11:32.681051 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.680471    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-workload-socket" pod "53accdd7-36ef-4487-812b-b0715588261d" (UID: "53accdd7-36ef-4487-812b-b0715588261d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:11:32.681051 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.680650    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-credential-socket" pod "53accdd7-36ef-4487-812b-b0715588261d" (UID: "53accdd7-36ef-4487-812b-b0715588261d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:11:32.681051 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.680813    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-credential-socket" pod "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" (UID: "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:11:32.681750 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.681313    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d05354b1-b462-4af4-af05-80c5a8c0de70-istiod-ca-cert" pod "d05354b1-b462-4af4-af05-80c5a8c0de70" (UID: "d05354b1-b462-4af4-af05-80c5a8c0de70"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:11:32.681750 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.681395    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-workload-socket" pod "d05354b1-b462-4af4-af05-80c5a8c0de70" (UID: "d05354b1-b462-4af4-af05-80c5a8c0de70"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:11:32.681750 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.681469    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-workload-certs" pod "d05354b1-b462-4af4-af05-80c5a8c0de70" (UID: "d05354b1-b462-4af4-af05-80c5a8c0de70"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:11:32.681750 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.681600    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-data" pod "d05354b1-b462-4af4-af05-80c5a8c0de70" (UID: "d05354b1-b462-4af4-af05-80c5a8c0de70"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:11:32.681750 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.681741    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-istio-data" pod "53accdd7-36ef-4487-812b-b0715588261d" (UID: "53accdd7-36ef-4487-812b-b0715588261d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:11:32.682232 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.682202    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53accdd7-36ef-4487-812b-b0715588261d-istiod-ca-cert" pod "53accdd7-36ef-4487-812b-b0715588261d" (UID: "53accdd7-36ef-4487-812b-b0715588261d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:11:32.682922 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.682894    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-envoy" pod "d05354b1-b462-4af4-af05-80c5a8c0de70" (UID: "d05354b1-b462-4af4-af05-80c5a8c0de70"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:11:32.683240 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.683201    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-envoy" pod "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" (UID: "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:11:32.683512 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.683484    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53accdd7-36ef-4487-812b-b0715588261d-kube-api-access-rmtgd" pod "53accdd7-36ef-4487-812b-b0715588261d" (UID: "53accdd7-36ef-4487-812b-b0715588261d"). InnerVolumeSpecName "kube-api-access-rmtgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:11:32.683652 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.683524    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d05354b1-b462-4af4-af05-80c5a8c0de70-kube-api-access-xxx2k" pod "d05354b1-b462-4af4-af05-80c5a8c0de70" (UID: "d05354b1-b462-4af4-af05-80c5a8c0de70"). InnerVolumeSpecName "kube-api-access-xxx2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:11:32.684014 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.683993    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-podinfo" pod "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" (UID: "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:11:32.684187 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.684169    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-podinfo" pod "d05354b1-b462-4af4-af05-80c5a8c0de70" (UID: "d05354b1-b462-4af4-af05-80c5a8c0de70"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:11:32.684764 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.684726    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-istio-envoy" pod "53accdd7-36ef-4487-812b-b0715588261d" (UID: "53accdd7-36ef-4487-812b-b0715588261d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:11:32.684857 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.684798    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-token" pod "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" (UID: "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:11:32.684857 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.684808    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/53accdd7-36ef-4487-812b-b0715588261d-istio-podinfo" pod "53accdd7-36ef-4487-812b-b0715588261d" (UID: "53accdd7-36ef-4487-812b-b0715588261d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:11:32.685060 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.685039    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53accdd7-36ef-4487-812b-b0715588261d-istio-token" pod "53accdd7-36ef-4487-812b-b0715588261d" (UID: "53accdd7-36ef-4487-812b-b0715588261d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:11:32.685202 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.685060    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-kube-api-access-jkxcx" pod "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" (UID: "c70d92a5-6ed0-470d-9c5d-822ffc97bbbc"). InnerVolumeSpecName "kube-api-access-jkxcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:11:32.686042 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.686024    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-token" pod "d05354b1-b462-4af4-af05-80c5a8c0de70" (UID: "d05354b1-b462-4af4-af05-80c5a8c0de70"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:11:32.780995 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.780903    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53accdd7-36ef-4487-812b-b0715588261d-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.780995 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.780938    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.780995 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.780953    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.780995 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.780966    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.780995 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.780981    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.780995 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.780993    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.781402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.781006    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53accdd7-36ef-4487-812b-b0715588261d-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.781402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.781018    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.781402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.781029    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.781402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.781041    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.781402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.781052    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.781402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.781064    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.781402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.781078    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmtgd\" (UniqueName: \"kubernetes.io/projected/53accdd7-36ef-4487-812b-b0715588261d-kube-api-access-rmtgd\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.781402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.781090    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53accdd7-36ef-4487-812b-b0715588261d-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.781402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.781102    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.781402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.781145    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkxcx\" (UniqueName: \"kubernetes.io/projected/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-kube-api-access-jkxcx\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.781402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.781159    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.781402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.781171    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53accdd7-36ef-4487-812b-b0715588261d-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.781402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.781183    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxx2k\" (UniqueName: \"kubernetes.io/projected/d05354b1-b462-4af4-af05-80c5a8c0de70-kube-api-access-xxx2k\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.781402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.781197    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d05354b1-b462-4af4-af05-80c5a8c0de70-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.781402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.781210    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:32.781402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:32.781223    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d05354b1-b462-4af4-af05-80c5a8c0de70-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:11:33.070092 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:33.070007    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"]
Nov 02 02:11:33.073877 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:33.073846    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-n72w9"]
Nov 02 02:11:33.182389 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:33.182351    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"]
Nov 02 02:11:33.186190 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:33.186155    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-xhmml"]
Nov 02 02:11:33.294961 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:33.294930    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"]
Nov 02 02:11:33.298835 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:33.298810    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5700d3d1-openshift-default-98f54c69-wqhlp"]
Nov 02 02:11:34.952648 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:34.952614    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="53accdd7-36ef-4487-812b-b0715588261d" path="/var/lib/kubelet/pods/53accdd7-36ef-4487-812b-b0715588261d/volumes"
Nov 02 02:11:34.953071 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:34.953058    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" path="/var/lib/kubelet/pods/c70d92a5-6ed0-470d-9c5d-822ffc97bbbc/volumes"
Nov 02 02:11:34.953506 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:34.953492    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d05354b1-b462-4af4-af05-80c5a8c0de70" path="/var/lib/kubelet/pods/d05354b1-b462-4af4-af05-80c5a8c0de70/volumes"
Nov 02 02:11:39.958724 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:11:39.958682    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:11:51.208585 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:51.208557    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 02:11:51.213900 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:11:51.213876    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 02:12:09.958733 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:12:09.958699    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:12:29.656059 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:29.656027    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"]
Nov 02 02:12:29.705694 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:29.705659    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"]
Nov 02 02:12:29.707332 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:29.707305    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"]
Nov 02 02:12:39.958104 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:12:39.958067    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:12:41.178899 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:41.178863    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"]
Nov 02 02:12:44.675607 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:44.675549    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" podUID="a471077b-3923-4355-8fda-135f0b8f038a" containerName="main" containerID="cri-o://be8fcea67f04eb8cbf4829881dd5e80f2f80f5d42140be456b641df9b84638a3" gracePeriod=45
Nov 02 02:12:44.724488 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:44.724454    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main" containerID="cri-o://17b1db1f67528e21c6dbe6900d72aff00af48cf3f4fe83be0dfb5a0dbdd7d1ef" gracePeriod=45
Nov 02 02:12:44.725963 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:44.725937    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" podUID="228f299b-731d-4375-93a9-fa777801b0a1" containerName="main" containerID="cri-o://245e32b57808dee6131e0e7217b67c3cb0f61e0fbbd02a39d821492eb0a224ed" gracePeriod=45
Nov 02 02:12:44.956600 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:44.956554    4515 generic.go:356] "Generic (PLEG): container finished" podID="228f299b-731d-4375-93a9-fa777801b0a1" containerID="245e32b57808dee6131e0e7217b67c3cb0f61e0fbbd02a39d821492eb0a224ed" exitCode=0
Nov 02 02:12:44.956745 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:44.956711    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" event={"ID":"228f299b-731d-4375-93a9-fa777801b0a1","Type":"ContainerDied","Data":"245e32b57808dee6131e0e7217b67c3cb0f61e0fbbd02a39d821492eb0a224ed"}
Nov 02 02:12:44.959324 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:44.959293    4515 generic.go:356] "Generic (PLEG): container finished" podID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerID="17b1db1f67528e21c6dbe6900d72aff00af48cf3f4fe83be0dfb5a0dbdd7d1ef" exitCode=0
Nov 02 02:12:44.959474 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:44.959371    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" event={"ID":"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3","Type":"ContainerDied","Data":"17b1db1f67528e21c6dbe6900d72aff00af48cf3f4fe83be0dfb5a0dbdd7d1ef"}
Nov 02 02:12:44.961182 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:44.961158    4515 generic.go:356] "Generic (PLEG): container finished" podID="a471077b-3923-4355-8fda-135f0b8f038a" containerID="be8fcea67f04eb8cbf4829881dd5e80f2f80f5d42140be456b641df9b84638a3" exitCode=0
Nov 02 02:12:44.961304 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:44.961238    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" event={"ID":"a471077b-3923-4355-8fda-135f0b8f038a","Type":"ContainerDied","Data":"be8fcea67f04eb8cbf4829881dd5e80f2f80f5d42140be456b641df9b84638a3"}
Nov 02 02:12:44.992875 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:44.992844    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 02:12:45.027495 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.027469    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 02:12:45.039376 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.039357    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 02:12:45.063422 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.063387    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-model-cache\") pod \"a471077b-3923-4355-8fda-135f0b8f038a\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") "
Nov 02 02:12:45.063620 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.063473    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a471077b-3923-4355-8fda-135f0b8f038a-tls-certs\" (UniqueName: \"kubernetes.io/secret/a471077b-3923-4355-8fda-135f0b8f038a-tls-certs\") pod \"a471077b-3923-4355-8fda-135f0b8f038a\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") "
Nov 02 02:12:45.063620 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.063517    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-tmp-dir\") pod \"a471077b-3923-4355-8fda-135f0b8f038a\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") "
Nov 02 02:12:45.063620 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.063573    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-dshm\") pod \"a471077b-3923-4355-8fda-135f0b8f038a\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") "
Nov 02 02:12:45.063620 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.063613    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a471077b-3923-4355-8fda-135f0b8f038a-kube-api-access-gnhbw\" (UniqueName: \"kubernetes.io/projected/a471077b-3923-4355-8fda-135f0b8f038a-kube-api-access-gnhbw\") pod \"a471077b-3923-4355-8fda-135f0b8f038a\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") "
Nov 02 02:12:45.063830 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.063677    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-home\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-home\") pod \"a471077b-3923-4355-8fda-135f0b8f038a\" (UID: \"a471077b-3923-4355-8fda-135f0b8f038a\") "
Nov 02 02:12:45.064586 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.064351    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-home" pod "a471077b-3923-4355-8fda-135f0b8f038a" (UID: "a471077b-3923-4355-8fda-135f0b8f038a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:12:45.064586 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.064540    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-model-cache" pod "a471077b-3923-4355-8fda-135f0b8f038a" (UID: "a471077b-3923-4355-8fda-135f0b8f038a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:12:45.065011 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.064981    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-tmp-dir" pod "a471077b-3923-4355-8fda-135f0b8f038a" (UID: "a471077b-3923-4355-8fda-135f0b8f038a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:12:45.068281 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.068234    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-dshm" pod "a471077b-3923-4355-8fda-135f0b8f038a" (UID: "a471077b-3923-4355-8fda-135f0b8f038a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:12:45.072202 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.069268    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a471077b-3923-4355-8fda-135f0b8f038a-tls-certs" pod "a471077b-3923-4355-8fda-135f0b8f038a" (UID: "a471077b-3923-4355-8fda-135f0b8f038a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:12:45.072202 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.069281    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a471077b-3923-4355-8fda-135f0b8f038a-kube-api-access-gnhbw" pod "a471077b-3923-4355-8fda-135f0b8f038a" (UID: "a471077b-3923-4355-8fda-135f0b8f038a"). InnerVolumeSpecName "kube-api-access-gnhbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:12:45.165197 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.165167    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-home\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-home\") pod \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") "
Nov 02 02:12:45.165389 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.165209    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-dshm\") pod \"228f299b-731d-4375-93a9-fa777801b0a1\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") "
Nov 02 02:12:45.165389 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.165258    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-model-cache\") pod \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") "
Nov 02 02:12:45.165389 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.165293    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/228f299b-731d-4375-93a9-fa777801b0a1-tls-certs\" (UniqueName: \"kubernetes.io/secret/228f299b-731d-4375-93a9-fa777801b0a1-tls-certs\") pod \"228f299b-731d-4375-93a9-fa777801b0a1\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") "
Nov 02 02:12:45.165389 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.165310    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-tls-certs\" (UniqueName: \"kubernetes.io/secret/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-tls-certs\") pod \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") "
Nov 02 02:12:45.165389 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.165335    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-dshm\") pod \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") "
Nov 02 02:12:45.165389 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.165364    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-model-cache\") pod \"228f299b-731d-4375-93a9-fa777801b0a1\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") "
Nov 02 02:12:45.165716 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.165430    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-kube-api-access-cm5j8\" (UniqueName: \"kubernetes.io/projected/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-kube-api-access-cm5j8\") pod \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") "
Nov 02 02:12:45.165716 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.165458    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-home" pod "f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" (UID: "f03b6e9e-52c5-47cd-9de5-2073fc55f0d3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:12:45.165716 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.165475    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-home\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-home\") pod \"228f299b-731d-4375-93a9-fa777801b0a1\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") "
Nov 02 02:12:45.165716 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.165559    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-model-cache" pod "f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" (UID: "f03b6e9e-52c5-47cd-9de5-2073fc55f0d3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:12:45.165716 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.165589    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/228f299b-731d-4375-93a9-fa777801b0a1-kube-api-access-h7zfl\" (UniqueName: \"kubernetes.io/projected/228f299b-731d-4375-93a9-fa777801b0a1-kube-api-access-h7zfl\") pod \"228f299b-731d-4375-93a9-fa777801b0a1\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") "
Nov 02 02:12:45.165716 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.165626    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-tmp-dir\") pod \"228f299b-731d-4375-93a9-fa777801b0a1\" (UID: \"228f299b-731d-4375-93a9-fa777801b0a1\") "
Nov 02 02:12:45.165716 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.165671    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-tmp-dir\") pod \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\" (UID: \"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3\") "
Nov 02 02:12:45.165716 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.165693    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-model-cache" pod "228f299b-731d-4375-93a9-fa777801b0a1" (UID: "228f299b-731d-4375-93a9-fa777801b0a1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:12:45.166133 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.165885    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-tmp-dir" pod "228f299b-731d-4375-93a9-fa777801b0a1" (UID: "228f299b-731d-4375-93a9-fa777801b0a1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:12:45.166133 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.166082    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-tmp-dir" pod "f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" (UID: "f03b6e9e-52c5-47cd-9de5-2073fc55f0d3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:12:45.166251 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.166188    4515 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-tmp-dir\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.166251 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.166208    4515 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-tmp-dir\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.166251 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.166222    4515 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-home\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.166251 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.166227    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-home" pod "228f299b-731d-4375-93a9-fa777801b0a1" (UID: "228f299b-731d-4375-93a9-fa777801b0a1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:12:45.166463 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.166237    4515 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-model-cache\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.166463 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.166282    4515 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-model-cache\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.166463 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.166302    4515 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a471077b-3923-4355-8fda-135f0b8f038a-tls-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.166463 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.166318    4515 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-model-cache\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.166463 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.166332    4515 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-tmp-dir\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.166463 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.166346    4515 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-dshm\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.166463 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.166361    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnhbw\" (UniqueName: \"kubernetes.io/projected/a471077b-3923-4355-8fda-135f0b8f038a-kube-api-access-gnhbw\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.166463 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.166377    4515 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a471077b-3923-4355-8fda-135f0b8f038a-home\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.167733 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.167703    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-dshm" pod "228f299b-731d-4375-93a9-fa777801b0a1" (UID: "228f299b-731d-4375-93a9-fa777801b0a1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:12:45.167857 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.167797    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-tls-certs" pod "f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" (UID: "f03b6e9e-52c5-47cd-9de5-2073fc55f0d3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:12:45.167943 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.167894    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-kube-api-access-cm5j8" pod "f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" (UID: "f03b6e9e-52c5-47cd-9de5-2073fc55f0d3"). InnerVolumeSpecName "kube-api-access-cm5j8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:12:45.167943 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.167929    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/228f299b-731d-4375-93a9-fa777801b0a1-tls-certs" pod "228f299b-731d-4375-93a9-fa777801b0a1" (UID: "228f299b-731d-4375-93a9-fa777801b0a1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:12:45.168226 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.168204    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-dshm" pod "f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" (UID: "f03b6e9e-52c5-47cd-9de5-2073fc55f0d3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:12:45.168304 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.168276    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/228f299b-731d-4375-93a9-fa777801b0a1-kube-api-access-h7zfl" pod "228f299b-731d-4375-93a9-fa777801b0a1" (UID: "228f299b-731d-4375-93a9-fa777801b0a1"). InnerVolumeSpecName "kube-api-access-h7zfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:12:45.268005 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.267975    4515 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-dshm\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.268005 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.268004    4515 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/228f299b-731d-4375-93a9-fa777801b0a1-tls-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.268255 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.268018    4515 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-tls-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.268255 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.268028    4515 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-dshm\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.268255 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.268042    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cm5j8\" (UniqueName: \"kubernetes.io/projected/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3-kube-api-access-cm5j8\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.268255 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.268057    4515 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/228f299b-731d-4375-93a9-fa777801b0a1-home\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.268255 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.268069    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7zfl\" (UniqueName: \"kubernetes.io/projected/228f299b-731d-4375-93a9-fa777801b0a1-kube-api-access-h7zfl\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:45.969984 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.969900    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"
Nov 02 02:12:45.969984 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.969932    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv" event={"ID":"228f299b-731d-4375-93a9-fa777801b0a1","Type":"ContainerDied","Data":"3783c3f13448b504092fea9e2e4c228c542bb76e369c4042faa7b8d4baf658c4"}
Nov 02 02:12:45.970539 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.969993    4515 scope.go:122] "RemoveContainer" containerID="245e32b57808dee6131e0e7217b67c3cb0f61e0fbbd02a39d821492eb0a224ed"
Nov 02 02:12:45.971329 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.971233    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"
Nov 02 02:12:45.971329 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.971241    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5" event={"ID":"f03b6e9e-52c5-47cd-9de5-2073fc55f0d3","Type":"ContainerDied","Data":"2f24820fea4503161952374c2901d5ad0c98259a1aaf31a4a4b33e77cb133bee"}
Nov 02 02:12:45.972617 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.972595    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5" event={"ID":"a471077b-3923-4355-8fda-135f0b8f038a","Type":"ContainerDied","Data":"bbcf22e51adfb276d1a5aab78598337d574eb2c88c0e09daea327083c581dddb"}
Nov 02 02:12:45.972719 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.972635    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"
Nov 02 02:12:45.978679 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.978664    4515 scope.go:122] "RemoveContainer" containerID="17b1db1f67528e21c6dbe6900d72aff00af48cf3f4fe83be0dfb5a0dbdd7d1ef"
Nov 02 02:12:45.988746 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:45.988730    4515 scope.go:122] "RemoveContainer" containerID="be8fcea67f04eb8cbf4829881dd5e80f2f80f5d42140be456b641df9b84638a3"
Nov 02 02:12:46.134519 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:46.134483    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"]
Nov 02 02:12:46.183890 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:46.183848    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/simulator-w0-9470a3-kserve-dd5f6b769-r2tqv"]
Nov 02 02:12:46.247851 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:46.247760    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"]
Nov 02 02:12:46.249626 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:46.249599    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/premium-w0-9470a3-kserve-64b6755468-rpxw5"]
Nov 02 02:12:46.357699 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:46.357667    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"]
Nov 02 02:12:46.362445 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:46.362417    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-9470a3/distinct-w0-9470a3-kserve-99f6f6d55-bqbz5"]
Nov 02 02:12:46.952418 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:46.952385    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="228f299b-731d-4375-93a9-fa777801b0a1" path="/var/lib/kubelet/pods/228f299b-731d-4375-93a9-fa777801b0a1/volumes"
Nov 02 02:12:46.952879 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:46.952864    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a471077b-3923-4355-8fda-135f0b8f038a" path="/var/lib/kubelet/pods/a471077b-3923-4355-8fda-135f0b8f038a/volumes"
Nov 02 02:12:46.953323 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:46.953308    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" path="/var/lib/kubelet/pods/f03b6e9e-52c5-47cd-9de5-2073fc55f0d3/volumes"
Nov 02 02:12:52.977307 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:52.977268    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-7wmnn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 02:12:52.977924 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:52.977889    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn" podUID="dd24d725-bfe3-4cde-9fc7-5d6d4e497995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 02:12:56.198771 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.198702    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main" containerID="cri-o://8b62a7277f60540b0cb20b025f942f2e087c792b1dded97b5ab3dab144408867" gracePeriod=45
Nov 02 02:12:56.442792 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.442766    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 02:12:56.541620 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.541529    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-tls-certs\" (UniqueName: \"kubernetes.io/secret/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-tls-certs\") pod \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") "
Nov 02 02:12:56.541620 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.541589    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-kube-api-access-ltmg5\" (UniqueName: \"kubernetes.io/projected/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-kube-api-access-ltmg5\") pod \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") "
Nov 02 02:12:56.541865 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.541632    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-model-cache\") pod \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") "
Nov 02 02:12:56.541865 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.541667    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-tmp-dir\") pod \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") "
Nov 02 02:12:56.541865 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.541693    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-home\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-home\") pod \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") "
Nov 02 02:12:56.541865 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.541743    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-dshm\") pod \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\" (UID: \"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc\") "
Nov 02 02:12:56.542074 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.541952    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-model-cache" pod "9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" (UID: "9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:12:56.542074 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.541965    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-tmp-dir" pod "9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" (UID: "9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:12:56.542074 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.542010    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-home" pod "9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" (UID: "9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:12:56.542249 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.542155    4515 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-model-cache\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:56.542249 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.542178    4515 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-tmp-dir\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:56.542249 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.542193    4515 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-home\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:56.543916 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.543887    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-tls-certs" pod "9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" (UID: "9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:12:56.544022 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.543962    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-kube-api-access-ltmg5" pod "9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" (UID: "9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc"). InnerVolumeSpecName "kube-api-access-ltmg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:12:56.544022 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.543972    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-dshm" pod "9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" (UID: "9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:12:56.642912 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.642869    4515 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-tls-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:56.642912 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.642907    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltmg5\" (UniqueName: \"kubernetes.io/projected/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-kube-api-access-ltmg5\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:56.642912 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:56.642919    4515 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc-dshm\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:12:57.038615 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:57.038578    4515 generic.go:356] "Generic (PLEG): container finished" podID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerID="8b62a7277f60540b0cb20b025f942f2e087c792b1dded97b5ab3dab144408867" exitCode=0
Nov 02 02:12:57.038810 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:57.038640    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"
Nov 02 02:12:57.038810 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:57.038662    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" event={"ID":"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc","Type":"ContainerDied","Data":"8b62a7277f60540b0cb20b025f942f2e087c792b1dded97b5ab3dab144408867"}
Nov 02 02:12:57.038810 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:57.038701    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" event={"ID":"9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc","Type":"ContainerDied","Data":"bc34c74efafd3a407c3108cac220e5dfeb4e33b3f238224992308944c2bf208d"}
Nov 02 02:12:57.038810 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:57.038723    4515 scope.go:122] "RemoveContainer" containerID="8b62a7277f60540b0cb20b025f942f2e087c792b1dded97b5ab3dab144408867"
Nov 02 02:12:57.047257 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:57.047234    4515 scope.go:122] "RemoveContainer" containerID="8b62a7277f60540b0cb20b025f942f2e087c792b1dded97b5ab3dab144408867"
Nov 02 02:12:57.047524 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:12:57.047502    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b62a7277f60540b0cb20b025f942f2e087c792b1dded97b5ab3dab144408867\": container with ID starting with 8b62a7277f60540b0cb20b025f942f2e087c792b1dded97b5ab3dab144408867 not found: ID does not exist" containerID="8b62a7277f60540b0cb20b025f942f2e087c792b1dded97b5ab3dab144408867"
Nov 02 02:12:57.047590 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:57.047538    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b62a7277f60540b0cb20b025f942f2e087c792b1dded97b5ab3dab144408867"} err="failed to get container status \"8b62a7277f60540b0cb20b025f942f2e087c792b1dded97b5ab3dab144408867\": rpc error: code = NotFound desc = could not find container \"8b62a7277f60540b0cb20b025f942f2e087c792b1dded97b5ab3dab144408867\": container with ID starting with 8b62a7277f60540b0cb20b025f942f2e087c792b1dded97b5ab3dab144408867 not found: ID does not exist"
Nov 02 02:12:57.202475 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:57.202435    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"]
Nov 02 02:12:57.208694 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:57.208664    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp"]
Nov 02 02:12:57.427558 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:57.427465    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-dbe849/simulator-w1-dbe849-kserve-698d9cf47b-cxhqp" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 02 02:12:58.952454 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:12:58.952423    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" path="/var/lib/kubelet/pods/9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc/volumes"
Nov 02 02:13:09.959088 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:13:09.959007    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:13:39.960265 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:13:39.960225    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:13:41.657975 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:41.657936    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"]
Nov 02 02:13:41.658502 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:41.658153    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl" podUID="de2ef1ec-9dc8-486e-949e-3d12cfdac897" containerName="maas-api" containerID="cri-o://baaa9e222edbc7014760a61f2ed5504b0756b7bae8d1a2f71dca40642843768d" gracePeriod=30
Nov 02 02:13:41.956831 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:41.956802    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 02:13:42.056519 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.056484    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/de2ef1ec-9dc8-486e-949e-3d12cfdac897-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de2ef1ec-9dc8-486e-949e-3d12cfdac897-maas-api-tls\") pod \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\" (UID: \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\") "
Nov 02 02:13:42.056721 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.056587    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de2ef1ec-9dc8-486e-949e-3d12cfdac897-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de2ef1ec-9dc8-486e-949e-3d12cfdac897-openshift-service-ca\") pod \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\" (UID: \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\") "
Nov 02 02:13:42.056721 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.056624    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de2ef1ec-9dc8-486e-949e-3d12cfdac897-kube-api-access-dxggv\" (UniqueName: \"kubernetes.io/projected/de2ef1ec-9dc8-486e-949e-3d12cfdac897-kube-api-access-dxggv\") pod \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\" (UID: \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\") "
Nov 02 02:13:42.056894 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.056780    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/de2ef1ec-9dc8-486e-949e-3d12cfdac897-metrics-certs\" (UniqueName: \"kubernetes.io/secret/de2ef1ec-9dc8-486e-949e-3d12cfdac897-metrics-certs\") pod \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\" (UID: \"de2ef1ec-9dc8-486e-949e-3d12cfdac897\") "
Nov 02 02:13:42.059532 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.059497    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de2ef1ec-9dc8-486e-949e-3d12cfdac897-maas-api-tls" pod "de2ef1ec-9dc8-486e-949e-3d12cfdac897" (UID: "de2ef1ec-9dc8-486e-949e-3d12cfdac897"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:13:42.061416 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.061371    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de2ef1ec-9dc8-486e-949e-3d12cfdac897-openshift-service-ca" pod "de2ef1ec-9dc8-486e-949e-3d12cfdac897" (UID: "de2ef1ec-9dc8-486e-949e-3d12cfdac897"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:13:42.065888 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.065855    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de2ef1ec-9dc8-486e-949e-3d12cfdac897-metrics-certs" pod "de2ef1ec-9dc8-486e-949e-3d12cfdac897" (UID: "de2ef1ec-9dc8-486e-949e-3d12cfdac897"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:13:42.068257 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.068223    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de2ef1ec-9dc8-486e-949e-3d12cfdac897-kube-api-access-dxggv" pod "de2ef1ec-9dc8-486e-949e-3d12cfdac897" (UID: "de2ef1ec-9dc8-486e-949e-3d12cfdac897"). InnerVolumeSpecName "kube-api-access-dxggv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:13:42.158689 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.158648    4515 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de2ef1ec-9dc8-486e-949e-3d12cfdac897-metrics-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:13:42.158689 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.158682    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de2ef1ec-9dc8-486e-949e-3d12cfdac897-maas-api-tls\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:13:42.158689 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.158693    4515 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de2ef1ec-9dc8-486e-949e-3d12cfdac897-openshift-service-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:13:42.158928 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.158702    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxggv\" (UniqueName: \"kubernetes.io/projected/de2ef1ec-9dc8-486e-949e-3d12cfdac897-kube-api-access-dxggv\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:13:42.253845 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.253811    4515 generic.go:356] "Generic (PLEG): container finished" podID="de2ef1ec-9dc8-486e-949e-3d12cfdac897" containerID="baaa9e222edbc7014760a61f2ed5504b0756b7bae8d1a2f71dca40642843768d" exitCode=0
Nov 02 02:13:42.254025 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.253868    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"
Nov 02 02:13:42.254025 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.253883    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl" event={"ID":"de2ef1ec-9dc8-486e-949e-3d12cfdac897","Type":"ContainerDied","Data":"baaa9e222edbc7014760a61f2ed5504b0756b7bae8d1a2f71dca40642843768d"}
Nov 02 02:13:42.254025 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.253926    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl" event={"ID":"de2ef1ec-9dc8-486e-949e-3d12cfdac897","Type":"ContainerDied","Data":"cfd02f5b5330e2a4124c924cf74cc2940a4b02ac4a165b23c0c3294a7f6854bf"}
Nov 02 02:13:42.254025 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.253948    4515 scope.go:122] "RemoveContainer" containerID="baaa9e222edbc7014760a61f2ed5504b0756b7bae8d1a2f71dca40642843768d"
Nov 02 02:13:42.264004 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.263980    4515 scope.go:122] "RemoveContainer" containerID="baaa9e222edbc7014760a61f2ed5504b0756b7bae8d1a2f71dca40642843768d"
Nov 02 02:13:42.264304 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:13:42.264284    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baaa9e222edbc7014760a61f2ed5504b0756b7bae8d1a2f71dca40642843768d\": container with ID starting with baaa9e222edbc7014760a61f2ed5504b0756b7bae8d1a2f71dca40642843768d not found: ID does not exist" containerID="baaa9e222edbc7014760a61f2ed5504b0756b7bae8d1a2f71dca40642843768d"
Nov 02 02:13:42.264390 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.264318    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baaa9e222edbc7014760a61f2ed5504b0756b7bae8d1a2f71dca40642843768d"} err="failed to get container status \"baaa9e222edbc7014760a61f2ed5504b0756b7bae8d1a2f71dca40642843768d\": rpc error: code = NotFound desc = could not find container \"baaa9e222edbc7014760a61f2ed5504b0756b7bae8d1a2f71dca40642843768d\": container with ID starting with baaa9e222edbc7014760a61f2ed5504b0756b7bae8d1a2f71dca40642843768d not found: ID does not exist"
Nov 02 02:13:42.421210 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.421168    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"]
Nov 02 02:13:42.424055 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.424022    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9470a3-76c45488c5-nzpzl"]
Nov 02 02:13:42.952878 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:42.952839    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de2ef1ec-9dc8-486e-949e-3d12cfdac897" path="/var/lib/kubelet/pods/de2ef1ec-9dc8-486e-949e-3d12cfdac897/volumes"
Nov 02 02:13:47.916741 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:47.916707    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"]
Nov 02 02:13:47.919207 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:47.916914    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt" podUID="b3cb7579-408d-42e0-8ca2-9972bfb54b84" containerName="maas-api" containerID="cri-o://4f1bc9b7618c8c5f0d8c9761cb179cf452eb9cb5b3545ada7ea6de3fcedc46b2" gracePeriod=30
Nov 02 02:13:48.223446 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.223297    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 02:13:48.284697 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.284665    4515 generic.go:356] "Generic (PLEG): container finished" podID="b3cb7579-408d-42e0-8ca2-9972bfb54b84" containerID="4f1bc9b7618c8c5f0d8c9761cb179cf452eb9cb5b3545ada7ea6de3fcedc46b2" exitCode=0
Nov 02 02:13:48.284885 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.284706    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt" event={"ID":"b3cb7579-408d-42e0-8ca2-9972bfb54b84","Type":"ContainerDied","Data":"4f1bc9b7618c8c5f0d8c9761cb179cf452eb9cb5b3545ada7ea6de3fcedc46b2"}
Nov 02 02:13:48.284885 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.284723    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"
Nov 02 02:13:48.284885 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.284752    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt" event={"ID":"b3cb7579-408d-42e0-8ca2-9972bfb54b84","Type":"ContainerDied","Data":"88e851f476d71ec9c37918ab369fdcb01d6b2996ba3cde476ca76883cf074024"}
Nov 02 02:13:48.284885 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.284775    4515 scope.go:122] "RemoveContainer" containerID="4f1bc9b7618c8c5f0d8c9761cb179cf452eb9cb5b3545ada7ea6de3fcedc46b2"
Nov 02 02:13:48.294168 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.294147    4515 scope.go:122] "RemoveContainer" containerID="4f1bc9b7618c8c5f0d8c9761cb179cf452eb9cb5b3545ada7ea6de3fcedc46b2"
Nov 02 02:13:48.294472 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:13:48.294452    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f1bc9b7618c8c5f0d8c9761cb179cf452eb9cb5b3545ada7ea6de3fcedc46b2\": container with ID starting with 4f1bc9b7618c8c5f0d8c9761cb179cf452eb9cb5b3545ada7ea6de3fcedc46b2 not found: ID does not exist" containerID="4f1bc9b7618c8c5f0d8c9761cb179cf452eb9cb5b3545ada7ea6de3fcedc46b2"
Nov 02 02:13:48.294520 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.294485    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f1bc9b7618c8c5f0d8c9761cb179cf452eb9cb5b3545ada7ea6de3fcedc46b2"} err="failed to get container status \"4f1bc9b7618c8c5f0d8c9761cb179cf452eb9cb5b3545ada7ea6de3fcedc46b2\": rpc error: code = NotFound desc = could not find container \"4f1bc9b7618c8c5f0d8c9761cb179cf452eb9cb5b3545ada7ea6de3fcedc46b2\": container with ID starting with 4f1bc9b7618c8c5f0d8c9761cb179cf452eb9cb5b3545ada7ea6de3fcedc46b2 not found: ID does not exist"
Nov 02 02:13:48.329735 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.328466    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b3cb7579-408d-42e0-8ca2-9972bfb54b84-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3cb7579-408d-42e0-8ca2-9972bfb54b84-maas-api-tls\") pod \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\" (UID: \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\") "
Nov 02 02:13:48.329735 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.328553    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3cb7579-408d-42e0-8ca2-9972bfb54b84-kube-api-access-lwhsg\" (UniqueName: \"kubernetes.io/projected/b3cb7579-408d-42e0-8ca2-9972bfb54b84-kube-api-access-lwhsg\") pod \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\" (UID: \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\") "
Nov 02 02:13:48.329735 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.328605    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b3cb7579-408d-42e0-8ca2-9972bfb54b84-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3cb7579-408d-42e0-8ca2-9972bfb54b84-openshift-service-ca\") pod \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\" (UID: \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\") "
Nov 02 02:13:48.329735 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.328753    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b3cb7579-408d-42e0-8ca2-9972bfb54b84-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3cb7579-408d-42e0-8ca2-9972bfb54b84-metrics-certs\") pod \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\" (UID: \"b3cb7579-408d-42e0-8ca2-9972bfb54b84\") "
Nov 02 02:13:48.329735 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.329502    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3cb7579-408d-42e0-8ca2-9972bfb54b84-openshift-service-ca" pod "b3cb7579-408d-42e0-8ca2-9972bfb54b84" (UID: "b3cb7579-408d-42e0-8ca2-9972bfb54b84"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:13:48.332968 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.332919    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3cb7579-408d-42e0-8ca2-9972bfb54b84-metrics-certs" pod "b3cb7579-408d-42e0-8ca2-9972bfb54b84" (UID: "b3cb7579-408d-42e0-8ca2-9972bfb54b84"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:13:48.333829 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.333792    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3cb7579-408d-42e0-8ca2-9972bfb54b84-maas-api-tls" pod "b3cb7579-408d-42e0-8ca2-9972bfb54b84" (UID: "b3cb7579-408d-42e0-8ca2-9972bfb54b84"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:13:48.334071 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.334048    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3cb7579-408d-42e0-8ca2-9972bfb54b84-kube-api-access-lwhsg" pod "b3cb7579-408d-42e0-8ca2-9972bfb54b84" (UID: "b3cb7579-408d-42e0-8ca2-9972bfb54b84"). InnerVolumeSpecName "kube-api-access-lwhsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:13:48.429920 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.429887    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3cb7579-408d-42e0-8ca2-9972bfb54b84-maas-api-tls\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:13:48.429920 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.429916    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwhsg\" (UniqueName: \"kubernetes.io/projected/b3cb7579-408d-42e0-8ca2-9972bfb54b84-kube-api-access-lwhsg\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:13:48.429920 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.429928    4515 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3cb7579-408d-42e0-8ca2-9972bfb54b84-openshift-service-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:13:48.430169 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.429937    4515 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3cb7579-408d-42e0-8ca2-9972bfb54b84-metrics-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:13:48.704503 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.704469    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"]
Nov 02 02:13:48.708608 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.708579    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-dbe849-589f9547f6-kjxxt"]
Nov 02 02:13:48.952947 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:48.952912    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3cb7579-408d-42e0-8ca2-9972bfb54b84" path="/var/lib/kubelet/pods/b3cb7579-408d-42e0-8ca2-9972bfb54b84/volumes"
Nov 02 02:13:51.763237 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:51.763200    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"]
Nov 02 02:13:51.763644 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:51.763400    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj" podUID="7ce83f3e-50cb-44f6-8c6c-4de081152950" containerName="istio-proxy" containerID="cri-o://4b964a75fed3cdd9c401ca110557b51f7fed522d9a4c33a99b8b1904a73c5a8f" gracePeriod=30
Nov 02 02:13:57.061040 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.061013    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 02:13:57.229918 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.229868    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-workload-certs\") pod \"7ce83f3e-50cb-44f6-8c6c-4de081152950\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") "
Nov 02 02:13:57.230233 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.229943    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-credential-socket\") pod \"7ce83f3e-50cb-44f6-8c6c-4de081152950\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") "
Nov 02 02:13:57.230233 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.230015    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-envoy\") pod \"7ce83f3e-50cb-44f6-8c6c-4de081152950\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") "
Nov 02 02:13:57.230233 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.230059    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-workload-socket\") pod \"7ce83f3e-50cb-44f6-8c6c-4de081152950\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") "
Nov 02 02:13:57.230233 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.230107    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-token\" (UniqueName: \"kubernetes.io/projected/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-token\") pod \"7ce83f3e-50cb-44f6-8c6c-4de081152950\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") "
Nov 02 02:13:57.230233 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.230168    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-podinfo\") pod \"7ce83f3e-50cb-44f6-8c6c-4de081152950\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") "
Nov 02 02:13:57.230233 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.230208    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ce83f3e-50cb-44f6-8c6c-4de081152950-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce83f3e-50cb-44f6-8c6c-4de081152950-istiod-ca-cert\") pod \"7ce83f3e-50cb-44f6-8c6c-4de081152950\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") "
Nov 02 02:13:57.230622 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.230256    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-data\") pod \"7ce83f3e-50cb-44f6-8c6c-4de081152950\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") "
Nov 02 02:13:57.230622 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.230262    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-workload-certs" pod "7ce83f3e-50cb-44f6-8c6c-4de081152950" (UID: "7ce83f3e-50cb-44f6-8c6c-4de081152950"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:13:57.230622 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.230286    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ce83f3e-50cb-44f6-8c6c-4de081152950-kube-api-access-rvssk\" (UniqueName: \"kubernetes.io/projected/7ce83f3e-50cb-44f6-8c6c-4de081152950-kube-api-access-rvssk\") pod \"7ce83f3e-50cb-44f6-8c6c-4de081152950\" (UID: \"7ce83f3e-50cb-44f6-8c6c-4de081152950\") "
Nov 02 02:13:57.230778 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.230695    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:13:57.231107 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.231076    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-credential-socket" pod "7ce83f3e-50cb-44f6-8c6c-4de081152950" (UID: "7ce83f3e-50cb-44f6-8c6c-4de081152950"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:13:57.231107 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.231090    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-data" pod "7ce83f3e-50cb-44f6-8c6c-4de081152950" (UID: "7ce83f3e-50cb-44f6-8c6c-4de081152950"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:13:57.231295 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.231103    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-workload-socket" pod "7ce83f3e-50cb-44f6-8c6c-4de081152950" (UID: "7ce83f3e-50cb-44f6-8c6c-4de081152950"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:13:57.231295 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.231272    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ce83f3e-50cb-44f6-8c6c-4de081152950-istiod-ca-cert" pod "7ce83f3e-50cb-44f6-8c6c-4de081152950" (UID: "7ce83f3e-50cb-44f6-8c6c-4de081152950"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:13:57.232968 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.232941    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-token" pod "7ce83f3e-50cb-44f6-8c6c-4de081152950" (UID: "7ce83f3e-50cb-44f6-8c6c-4de081152950"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:13:57.233282 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.233254    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce83f3e-50cb-44f6-8c6c-4de081152950-kube-api-access-rvssk" pod "7ce83f3e-50cb-44f6-8c6c-4de081152950" (UID: "7ce83f3e-50cb-44f6-8c6c-4de081152950"). InnerVolumeSpecName "kube-api-access-rvssk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:13:57.234521 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.234483    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-envoy" pod "7ce83f3e-50cb-44f6-8c6c-4de081152950" (UID: "7ce83f3e-50cb-44f6-8c6c-4de081152950"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:13:57.234521 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.234492    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-podinfo" pod "7ce83f3e-50cb-44f6-8c6c-4de081152950" (UID: "7ce83f3e-50cb-44f6-8c6c-4de081152950"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:13:57.328265 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.328184    4515 generic.go:356] "Generic (PLEG): container finished" podID="7ce83f3e-50cb-44f6-8c6c-4de081152950" containerID="4b964a75fed3cdd9c401ca110557b51f7fed522d9a4c33a99b8b1904a73c5a8f" exitCode=0
Nov 02 02:13:57.328265 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.328235    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"
Nov 02 02:13:57.328426 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.328237    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj" event={"ID":"7ce83f3e-50cb-44f6-8c6c-4de081152950","Type":"ContainerDied","Data":"4b964a75fed3cdd9c401ca110557b51f7fed522d9a4c33a99b8b1904a73c5a8f"}
Nov 02 02:13:57.328426 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.328365    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj" event={"ID":"7ce83f3e-50cb-44f6-8c6c-4de081152950","Type":"ContainerDied","Data":"60e7051f63284b5ba16b15f2a84f30a4203fe207f3673a451cb538543a74a32d"}
Nov 02 02:13:57.328426 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.328397    4515 scope.go:122] "RemoveContainer" containerID="4b964a75fed3cdd9c401ca110557b51f7fed522d9a4c33a99b8b1904a73c5a8f"
Nov 02 02:13:57.331293 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.331273    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:13:57.331293 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.331294    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:13:57.331442 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.331305    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:13:57.331442 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.331314    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:13:57.331442 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.331323    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:13:57.331442 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.331331    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce83f3e-50cb-44f6-8c6c-4de081152950-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:13:57.331442 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.331343    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce83f3e-50cb-44f6-8c6c-4de081152950-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:13:57.331442 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.331353    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvssk\" (UniqueName: \"kubernetes.io/projected/7ce83f3e-50cb-44f6-8c6c-4de081152950-kube-api-access-rvssk\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:13:57.338052 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.338032    4515 scope.go:122] "RemoveContainer" containerID="4b964a75fed3cdd9c401ca110557b51f7fed522d9a4c33a99b8b1904a73c5a8f"
Nov 02 02:13:57.338344 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:13:57.338325    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b964a75fed3cdd9c401ca110557b51f7fed522d9a4c33a99b8b1904a73c5a8f\": container with ID starting with 4b964a75fed3cdd9c401ca110557b51f7fed522d9a4c33a99b8b1904a73c5a8f not found: ID does not exist" containerID="4b964a75fed3cdd9c401ca110557b51f7fed522d9a4c33a99b8b1904a73c5a8f"
Nov 02 02:13:57.338398 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.338355    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b964a75fed3cdd9c401ca110557b51f7fed522d9a4c33a99b8b1904a73c5a8f"} err="failed to get container status \"4b964a75fed3cdd9c401ca110557b51f7fed522d9a4c33a99b8b1904a73c5a8f\": rpc error: code = NotFound desc = could not find container \"4b964a75fed3cdd9c401ca110557b51f7fed522d9a4c33a99b8b1904a73c5a8f\": container with ID starting with 4b964a75fed3cdd9c401ca110557b51f7fed522d9a4c33a99b8b1904a73c5a8f not found: ID does not exist"
Nov 02 02:13:57.492345 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.492310    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"]
Nov 02 02:13:57.496753 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:57.496727    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-9470a3-openshift-default-6769c45f77-22xvj"]
Nov 02 02:13:58.951965 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:13:58.951931    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ce83f3e-50cb-44f6-8c6c-4de081152950" path="/var/lib/kubelet/pods/7ce83f3e-50cb-44f6-8c6c-4de081152950/volumes"
Nov 02 02:14:09.959027 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:14:09.958985    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:14:17.801534 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:17.801498    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"]
Nov 02 02:14:17.802071 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:17.801703    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf" podUID="65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" containerName="istio-proxy" containerID="cri-o://9ef60c8f7803205461e1863dd91d08e9441f54b13468b9423a005c28bc6893b9" gracePeriod=30
Nov 02 02:14:23.102532 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.102506    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 02:14:23.190085 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.190051    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-data\") pod \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") "
Nov 02 02:14:23.190315 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.190112    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istiod-ca-cert\") pod \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") "
Nov 02 02:14:23.190315 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.190234    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-credential-socket\") pod \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") "
Nov 02 02:14:23.190315 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.190294    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-envoy\") pod \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") "
Nov 02 02:14:23.190488 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.190321    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-podinfo\") pod \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") "
Nov 02 02:14:23.190488 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.190385    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-workload-socket\") pod \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") "
Nov 02 02:14:23.190488 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.190440    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-data" pod "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" (UID: "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:14:23.190488 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.190477    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-token\" (UniqueName: \"kubernetes.io/projected/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-token\") pod \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") "
Nov 02 02:14:23.190684 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.190508    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-workload-certs\") pod \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") "
Nov 02 02:14:23.190684 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.190553    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-kube-api-access-qql7b\" (UniqueName: \"kubernetes.io/projected/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-kube-api-access-qql7b\") pod \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\" (UID: \"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9\") "
Nov 02 02:14:23.190909 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.190891    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:23.190981 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.190942    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-workload-socket" pod "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" (UID: "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:14:23.191580 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.191532    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-credential-socket" pod "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" (UID: "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:14:23.191580 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.191551    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istiod-ca-cert" pod "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" (UID: "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:14:23.192690 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.192664    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-workload-certs" pod "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" (UID: "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:14:23.193255 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.193230    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-podinfo" pod "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" (UID: "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:14:23.193657 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.193634    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-token" pod "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" (UID: "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:14:23.198368 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.198342    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-envoy" pod "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" (UID: "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:14:23.200062 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.200044    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-kube-api-access-qql7b" pod "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" (UID: "65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9"). InnerVolumeSpecName "kube-api-access-qql7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:14:23.291835 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.291804    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:23.291835 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.291832    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:23.291835 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.291842    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:23.292053 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.291851    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:23.292053 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.291860    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:23.292053 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.291870    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:23.292053 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.291879    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:23.292053 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.291888    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qql7b\" (UniqueName: \"kubernetes.io/projected/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9-kube-api-access-qql7b\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:23.447289 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.447248    4515 generic.go:356] "Generic (PLEG): container finished" podID="65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" containerID="9ef60c8f7803205461e1863dd91d08e9441f54b13468b9423a005c28bc6893b9" exitCode=0
Nov 02 02:14:23.447487 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.447294    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf" event={"ID":"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9","Type":"ContainerDied","Data":"9ef60c8f7803205461e1863dd91d08e9441f54b13468b9423a005c28bc6893b9"}
Nov 02 02:14:23.447487 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.447312    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"
Nov 02 02:14:23.447487 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.447361    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf" event={"ID":"65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9","Type":"ContainerDied","Data":"c3bcdcfc45324cab9ed949d1c31825dece5ccebb0cbbb65ce6146581a2636fe5"}
Nov 02 02:14:23.447487 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.447382    4515 scope.go:122] "RemoveContainer" containerID="9ef60c8f7803205461e1863dd91d08e9441f54b13468b9423a005c28bc6893b9"
Nov 02 02:14:23.456398 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.456378    4515 scope.go:122] "RemoveContainer" containerID="9ef60c8f7803205461e1863dd91d08e9441f54b13468b9423a005c28bc6893b9"
Nov 02 02:14:23.456658 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:14:23.456640    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef60c8f7803205461e1863dd91d08e9441f54b13468b9423a005c28bc6893b9\": container with ID starting with 9ef60c8f7803205461e1863dd91d08e9441f54b13468b9423a005c28bc6893b9 not found: ID does not exist" containerID="9ef60c8f7803205461e1863dd91d08e9441f54b13468b9423a005c28bc6893b9"
Nov 02 02:14:23.456706 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.456669    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef60c8f7803205461e1863dd91d08e9441f54b13468b9423a005c28bc6893b9"} err="failed to get container status \"9ef60c8f7803205461e1863dd91d08e9441f54b13468b9423a005c28bc6893b9\": rpc error: code = NotFound desc = could not find container \"9ef60c8f7803205461e1863dd91d08e9441f54b13468b9423a005c28bc6893b9\": container with ID starting with 9ef60c8f7803205461e1863dd91d08e9441f54b13468b9423a005c28bc6893b9 not found: ID does not exist"
Nov 02 02:14:23.611541 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.611508    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"]
Nov 02 02:14:23.615740 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:23.615716    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-0ebdb3-openshift-default-769ffb6bd5-jmstf"]
Nov 02 02:14:24.952321 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:24.952286    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" path="/var/lib/kubelet/pods/65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9/volumes"
Nov 02 02:14:27.094290 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.094255    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"]
Nov 02 02:14:27.094798 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.094547    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb" podUID="2f990ce6-351c-4a76-87b9-308d20c1b828" containerName="discovery" containerID="cri-o://6c19547249532be0904f59eb0bf824ce93c62053ccda32b2347e643167aef52b" gracePeriod=30
Nov 02 02:14:27.353853 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.353793    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 02:14:27.438712 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.438677    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-cacerts\" (UniqueName: \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-cacerts\") pod \"2f990ce6-351c-4a76-87b9-308d20c1b828\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") "
Nov 02 02:14:27.438895 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.438758    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f990ce6-351c-4a76-87b9-308d20c1b828-kube-api-access-g8pkj\" (UniqueName: \"kubernetes.io/projected/2f990ce6-351c-4a76-87b9-308d20c1b828-kube-api-access-g8pkj\") pod \"2f990ce6-351c-4a76-87b9-308d20c1b828\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") "
Nov 02 02:14:27.438967 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.438933    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-csr-dns-cert\") pod \"2f990ce6-351c-4a76-87b9-308d20c1b828\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") "
Nov 02 02:14:27.439024 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.438974    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-csr-ca-configmap\") pod \"2f990ce6-351c-4a76-87b9-308d20c1b828\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") "
Nov 02 02:14:27.439073 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.439035    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-kubeconfig\") pod \"2f990ce6-351c-4a76-87b9-308d20c1b828\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") "
Nov 02 02:14:27.439155 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.439100    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f990ce6-351c-4a76-87b9-308d20c1b828-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f990ce6-351c-4a76-87b9-308d20c1b828-local-certs\") pod \"2f990ce6-351c-4a76-87b9-308d20c1b828\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") "
Nov 02 02:14:27.439217 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.439173    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-token\" (UniqueName: \"kubernetes.io/projected/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-token\") pod \"2f990ce6-351c-4a76-87b9-308d20c1b828\" (UID: \"2f990ce6-351c-4a76-87b9-308d20c1b828\") "
Nov 02 02:14:27.439488 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.439416    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-csr-ca-configmap" pod "2f990ce6-351c-4a76-87b9-308d20c1b828" (UID: "2f990ce6-351c-4a76-87b9-308d20c1b828"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:14:27.439615 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.439576    4515 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-csr-ca-configmap\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:27.441254 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.441217    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-csr-dns-cert" pod "2f990ce6-351c-4a76-87b9-308d20c1b828" (UID: "2f990ce6-351c-4a76-87b9-308d20c1b828"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:14:27.441390 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.441254    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-kubeconfig" pod "2f990ce6-351c-4a76-87b9-308d20c1b828" (UID: "2f990ce6-351c-4a76-87b9-308d20c1b828"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:14:27.441462 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.441443    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-cacerts" pod "2f990ce6-351c-4a76-87b9-308d20c1b828" (UID: "2f990ce6-351c-4a76-87b9-308d20c1b828"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:14:27.441568 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.441543    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f990ce6-351c-4a76-87b9-308d20c1b828-local-certs" pod "2f990ce6-351c-4a76-87b9-308d20c1b828" (UID: "2f990ce6-351c-4a76-87b9-308d20c1b828"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:14:27.441658 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.441638    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f990ce6-351c-4a76-87b9-308d20c1b828-kube-api-access-g8pkj" pod "2f990ce6-351c-4a76-87b9-308d20c1b828" (UID: "2f990ce6-351c-4a76-87b9-308d20c1b828"). InnerVolumeSpecName "kube-api-access-g8pkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:14:27.441716 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.441666    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-token" pod "2f990ce6-351c-4a76-87b9-308d20c1b828" (UID: "2f990ce6-351c-4a76-87b9-308d20c1b828"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:14:27.466907 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.466880    4515 generic.go:356] "Generic (PLEG): container finished" podID="2f990ce6-351c-4a76-87b9-308d20c1b828" containerID="6c19547249532be0904f59eb0bf824ce93c62053ccda32b2347e643167aef52b" exitCode=0
Nov 02 02:14:27.467022 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.466926    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"
Nov 02 02:14:27.467022 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.466955    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb" event={"ID":"2f990ce6-351c-4a76-87b9-308d20c1b828","Type":"ContainerDied","Data":"6c19547249532be0904f59eb0bf824ce93c62053ccda32b2347e643167aef52b"}
Nov 02 02:14:27.467022 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.467000    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb" event={"ID":"2f990ce6-351c-4a76-87b9-308d20c1b828","Type":"ContainerDied","Data":"cdb1401dfb8242b8093f1e53c07444c981eb08445cd2c8f61ab4909eb9093b92"}
Nov 02 02:14:27.467154 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.467023    4515 scope.go:122] "RemoveContainer" containerID="6c19547249532be0904f59eb0bf824ce93c62053ccda32b2347e643167aef52b"
Nov 02 02:14:27.477047 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.477030    4515 scope.go:122] "RemoveContainer" containerID="6c19547249532be0904f59eb0bf824ce93c62053ccda32b2347e643167aef52b"
Nov 02 02:14:27.477400 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:14:27.477354    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c19547249532be0904f59eb0bf824ce93c62053ccda32b2347e643167aef52b\": container with ID starting with 6c19547249532be0904f59eb0bf824ce93c62053ccda32b2347e643167aef52b not found: ID does not exist" containerID="6c19547249532be0904f59eb0bf824ce93c62053ccda32b2347e643167aef52b"
Nov 02 02:14:27.477523 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.477396    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c19547249532be0904f59eb0bf824ce93c62053ccda32b2347e643167aef52b"} err="failed to get container status \"6c19547249532be0904f59eb0bf824ce93c62053ccda32b2347e643167aef52b\": rpc error: code = NotFound desc = could not find container \"6c19547249532be0904f59eb0bf824ce93c62053ccda32b2347e643167aef52b\": container with ID starting with 6c19547249532be0904f59eb0bf824ce93c62053ccda32b2347e643167aef52b not found: ID does not exist"
Nov 02 02:14:27.540367 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.540333    4515 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f990ce6-351c-4a76-87b9-308d20c1b828-local-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:27.540367 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.540358    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:27.540367 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.540370    4515 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-cacerts\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:27.540612 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.540380    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8pkj\" (UniqueName: \"kubernetes.io/projected/2f990ce6-351c-4a76-87b9-308d20c1b828-kube-api-access-g8pkj\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:27.540612 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.540391    4515 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-csr-dns-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:27.540612 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.540399    4515 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2f990ce6-351c-4a76-87b9-308d20c1b828-istio-kubeconfig\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:27.588073 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.588041    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"]
Nov 02 02:14:27.592341 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:27.592315    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zwstb"]
Nov 02 02:14:28.951716 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:28.951681    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f990ce6-351c-4a76-87b9-308d20c1b828" path="/var/lib/kubelet/pods/2f990ce6-351c-4a76-87b9-308d20c1b828/volumes"
Nov 02 02:14:37.417881 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:37.417789    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"]
Nov 02 02:14:37.468636 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:37.468597    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"]
Nov 02 02:14:39.959083 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:14:39.959044    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:14:52.437026 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.436983    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc" podUID="dc4865ed-8a4f-4432-afc1-61d816b83090" containerName="main" containerID="cri-o://4da28877e92beda49c4caf64ecca0f592ea37b2bd6960ea3e148d19bc8aa08bb" gracePeriod=45
Nov 02 02:14:52.489687 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.489643    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main" containerID="cri-o://e66c14ccb9856632233bab056368e61a0b3a4bed1e269e293a44e5e2b2b2e900" gracePeriod=45
Nov 02 02:14:52.579207 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.579176    4515 generic.go:356] "Generic (PLEG): container finished" podID="dc4865ed-8a4f-4432-afc1-61d816b83090" containerID="4da28877e92beda49c4caf64ecca0f592ea37b2bd6960ea3e148d19bc8aa08bb" exitCode=0
Nov 02 02:14:52.579374 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.579246    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc" event={"ID":"dc4865ed-8a4f-4432-afc1-61d816b83090","Type":"ContainerDied","Data":"4da28877e92beda49c4caf64ecca0f592ea37b2bd6960ea3e148d19bc8aa08bb"}
Nov 02 02:14:52.740746 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.740722    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 02:14:52.762097 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.762070    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 02:14:52.783967 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.783939    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-dshm\") pod \"dc4865ed-8a4f-4432-afc1-61d816b83090\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") "
Nov 02 02:14:52.784099 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.784000    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc4865ed-8a4f-4432-afc1-61d816b83090-tls-certs\" (UniqueName: \"kubernetes.io/secret/dc4865ed-8a4f-4432-afc1-61d816b83090-tls-certs\") pod \"dc4865ed-8a4f-4432-afc1-61d816b83090\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") "
Nov 02 02:14:52.784099 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.784059    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc4865ed-8a4f-4432-afc1-61d816b83090-kube-api-access-sxpvr\" (UniqueName: \"kubernetes.io/projected/dc4865ed-8a4f-4432-afc1-61d816b83090-kube-api-access-sxpvr\") pod \"dc4865ed-8a4f-4432-afc1-61d816b83090\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") "
Nov 02 02:14:52.784099 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.784096    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-tmp-dir\") pod \"dc4865ed-8a4f-4432-afc1-61d816b83090\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") "
Nov 02 02:14:52.784282 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.784140    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-model-cache\") pod \"dc4865ed-8a4f-4432-afc1-61d816b83090\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") "
Nov 02 02:14:52.784282 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.784194    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-home\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-home\") pod \"dc4865ed-8a4f-4432-afc1-61d816b83090\" (UID: \"dc4865ed-8a4f-4432-afc1-61d816b83090\") "
Nov 02 02:14:52.784411 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.784387    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-tmp-dir" pod "dc4865ed-8a4f-4432-afc1-61d816b83090" (UID: "dc4865ed-8a4f-4432-afc1-61d816b83090"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:14:52.784509 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.784482    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-model-cache" pod "dc4865ed-8a4f-4432-afc1-61d816b83090" (UID: "dc4865ed-8a4f-4432-afc1-61d816b83090"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:14:52.784559 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.784518    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-home" pod "dc4865ed-8a4f-4432-afc1-61d816b83090" (UID: "dc4865ed-8a4f-4432-afc1-61d816b83090"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:14:52.784653 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.784636    4515 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-tmp-dir\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:52.784706 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.784659    4515 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-model-cache\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:52.784706 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.784674    4515 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-home\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:52.786110 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.786092    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-dshm" pod "dc4865ed-8a4f-4432-afc1-61d816b83090" (UID: "dc4865ed-8a4f-4432-afc1-61d816b83090"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:14:52.786202 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.786162    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc4865ed-8a4f-4432-afc1-61d816b83090-tls-certs" pod "dc4865ed-8a4f-4432-afc1-61d816b83090" (UID: "dc4865ed-8a4f-4432-afc1-61d816b83090"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:14:52.786202 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.786170    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc4865ed-8a4f-4432-afc1-61d816b83090-kube-api-access-sxpvr" pod "dc4865ed-8a4f-4432-afc1-61d816b83090" (UID: "dc4865ed-8a4f-4432-afc1-61d816b83090"). InnerVolumeSpecName "kube-api-access-sxpvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:14:52.885366 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.885332    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-model-cache\") pod \"73a93fb2-af95-4246-8032-e8ce2025bbdd\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") "
Nov 02 02:14:52.885554 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.885399    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/73a93fb2-af95-4246-8032-e8ce2025bbdd-tls-certs\" (UniqueName: \"kubernetes.io/secret/73a93fb2-af95-4246-8032-e8ce2025bbdd-tls-certs\") pod \"73a93fb2-af95-4246-8032-e8ce2025bbdd\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") "
Nov 02 02:14:52.885554 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.885428    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-home\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-home\") pod \"73a93fb2-af95-4246-8032-e8ce2025bbdd\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") "
Nov 02 02:14:52.885554 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.885486    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-dshm\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-dshm\") pod \"73a93fb2-af95-4246-8032-e8ce2025bbdd\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") "
Nov 02 02:14:52.885554 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.885516    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73a93fb2-af95-4246-8032-e8ce2025bbdd-kube-api-access-kwph4\" (UniqueName: \"kubernetes.io/projected/73a93fb2-af95-4246-8032-e8ce2025bbdd-kube-api-access-kwph4\") pod \"73a93fb2-af95-4246-8032-e8ce2025bbdd\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") "
Nov 02 02:14:52.885783 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.885573    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-tmp-dir\") pod \"73a93fb2-af95-4246-8032-e8ce2025bbdd\" (UID: \"73a93fb2-af95-4246-8032-e8ce2025bbdd\") "
Nov 02 02:14:52.885783 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.885603    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-model-cache" pod "73a93fb2-af95-4246-8032-e8ce2025bbdd" (UID: "73a93fb2-af95-4246-8032-e8ce2025bbdd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:14:52.885783 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.885722    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-home" pod "73a93fb2-af95-4246-8032-e8ce2025bbdd" (UID: "73a93fb2-af95-4246-8032-e8ce2025bbdd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:14:52.885932 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.885842    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-tmp-dir" pod "73a93fb2-af95-4246-8032-e8ce2025bbdd" (UID: "73a93fb2-af95-4246-8032-e8ce2025bbdd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:14:52.885932 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.885906    4515 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-home\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:52.885932 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.885921    4515 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc4865ed-8a4f-4432-afc1-61d816b83090-dshm\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:52.885932 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.885930    4515 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc4865ed-8a4f-4432-afc1-61d816b83090-tls-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:52.886090 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.885941    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxpvr\" (UniqueName: \"kubernetes.io/projected/dc4865ed-8a4f-4432-afc1-61d816b83090-kube-api-access-sxpvr\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:52.886090 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.885951    4515 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-model-cache\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:52.887637 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.887611    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-dshm" pod "73a93fb2-af95-4246-8032-e8ce2025bbdd" (UID: "73a93fb2-af95-4246-8032-e8ce2025bbdd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:14:52.887637 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.887625    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73a93fb2-af95-4246-8032-e8ce2025bbdd-tls-certs" pod "73a93fb2-af95-4246-8032-e8ce2025bbdd" (UID: "73a93fb2-af95-4246-8032-e8ce2025bbdd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:14:52.887784 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.887644    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a93fb2-af95-4246-8032-e8ce2025bbdd-kube-api-access-kwph4" pod "73a93fb2-af95-4246-8032-e8ce2025bbdd" (UID: "73a93fb2-af95-4246-8032-e8ce2025bbdd"). InnerVolumeSpecName "kube-api-access-kwph4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:14:52.986541 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.986477    4515 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-dshm\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:52.986541 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.986502    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwph4\" (UniqueName: \"kubernetes.io/projected/73a93fb2-af95-4246-8032-e8ce2025bbdd-kube-api-access-kwph4\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:52.986541 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.986511    4515 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73a93fb2-af95-4246-8032-e8ce2025bbdd-tmp-dir\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:52.986541 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:52.986520    4515 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73a93fb2-af95-4246-8032-e8ce2025bbdd-tls-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:14:53.584252 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:53.584217    4515 generic.go:356] "Generic (PLEG): container finished" podID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerID="e66c14ccb9856632233bab056368e61a0b3a4bed1e269e293a44e5e2b2b2e900" exitCode=0
Nov 02 02:14:53.584718 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:53.584277    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"
Nov 02 02:14:53.584718 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:53.584300    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" event={"ID":"73a93fb2-af95-4246-8032-e8ce2025bbdd","Type":"ContainerDied","Data":"e66c14ccb9856632233bab056368e61a0b3a4bed1e269e293a44e5e2b2b2e900"}
Nov 02 02:14:53.584718 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:53.584341    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62" event={"ID":"73a93fb2-af95-4246-8032-e8ce2025bbdd","Type":"ContainerDied","Data":"2d4671f6292d5a6188e739380e13ffeea2841cc5d1f409987fd2bea481415614"}
Nov 02 02:14:53.584718 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:53.584360    4515 scope.go:122] "RemoveContainer" containerID="e66c14ccb9856632233bab056368e61a0b3a4bed1e269e293a44e5e2b2b2e900"
Nov 02 02:14:53.585791 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:53.585765    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc" event={"ID":"dc4865ed-8a4f-4432-afc1-61d816b83090","Type":"ContainerDied","Data":"c2ca3b6a2a5168f78499d83fbd4c0cb8c7f101a202679373394916cf61d8cf43"}
Nov 02 02:14:53.585791 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:53.585790    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"
Nov 02 02:14:53.592923 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:53.592905    4515 scope.go:122] "RemoveContainer" containerID="e66c14ccb9856632233bab056368e61a0b3a4bed1e269e293a44e5e2b2b2e900"
Nov 02 02:14:53.593249 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:14:53.593216    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e66c14ccb9856632233bab056368e61a0b3a4bed1e269e293a44e5e2b2b2e900\": container with ID starting with e66c14ccb9856632233bab056368e61a0b3a4bed1e269e293a44e5e2b2b2e900 not found: ID does not exist" containerID="e66c14ccb9856632233bab056368e61a0b3a4bed1e269e293a44e5e2b2b2e900"
Nov 02 02:14:53.593305 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:53.593262    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e66c14ccb9856632233bab056368e61a0b3a4bed1e269e293a44e5e2b2b2e900"} err="failed to get container status \"e66c14ccb9856632233bab056368e61a0b3a4bed1e269e293a44e5e2b2b2e900\": rpc error: code = NotFound desc = could not find container \"e66c14ccb9856632233bab056368e61a0b3a4bed1e269e293a44e5e2b2b2e900\": container with ID starting with e66c14ccb9856632233bab056368e61a0b3a4bed1e269e293a44e5e2b2b2e900 not found: ID does not exist"
Nov 02 02:14:53.593305 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:53.593280    4515 scope.go:122] "RemoveContainer" containerID="4da28877e92beda49c4caf64ecca0f592ea37b2bd6960ea3e148d19bc8aa08bb"
Nov 02 02:14:53.747178 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:53.747086    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"]
Nov 02 02:14:53.796526 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:53.796491    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-c62671/premium-w2-c62671-kserve-74bcdbcf87-57wvc"]
Nov 02 02:14:53.859181 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:53.859146    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"]
Nov 02 02:14:53.863619 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:53.863597    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-c62671/simulator-w2-c62671-kserve-78d86c9b54-rjh62"]
Nov 02 02:14:54.951783 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:54.951750    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" path="/var/lib/kubelet/pods/73a93fb2-af95-4246-8032-e8ce2025bbdd/volumes"
Nov 02 02:14:54.952183 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:14:54.952162    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc4865ed-8a4f-4432-afc1-61d816b83090" path="/var/lib/kubelet/pods/dc4865ed-8a4f-4432-afc1-61d816b83090/volumes"
Nov 02 02:15:00.162876 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.162842    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9"]
Nov 02 02:15:00.163280 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163200    4515 cpu_manager.go:414] "containerMap: removing container" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main"
Nov 02 02:15:00.163280 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163211    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main"
Nov 02 02:15:00.163280 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163223    4515 cpu_manager.go:414] "containerMap: removing container" podUID="2f990ce6-351c-4a76-87b9-308d20c1b828" containerName="discovery"
Nov 02 02:15:00.163280 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163228    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f990ce6-351c-4a76-87b9-308d20c1b828" containerName="discovery"
Nov 02 02:15:00.163280 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163242    4515 cpu_manager.go:414] "containerMap: removing container" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main"
Nov 02 02:15:00.163280 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163247    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main"
Nov 02 02:15:00.163280 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163258    4515 cpu_manager.go:414] "containerMap: removing container" podUID="de2ef1ec-9dc8-486e-949e-3d12cfdac897" containerName="maas-api"
Nov 02 02:15:00.163280 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163262    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de2ef1ec-9dc8-486e-949e-3d12cfdac897" containerName="maas-api"
Nov 02 02:15:00.163280 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163269    4515 cpu_manager.go:414] "containerMap: removing container" podUID="dc4865ed-8a4f-4432-afc1-61d816b83090" containerName="main"
Nov 02 02:15:00.163280 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163273    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc4865ed-8a4f-4432-afc1-61d816b83090" containerName="main"
Nov 02 02:15:00.163280 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163279    4515 cpu_manager.go:414] "containerMap: removing container" podUID="b05cc018-433c-4f3a-9708-95e3bdde50ff" containerName="istio-proxy"
Nov 02 02:15:00.163280 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163283    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b05cc018-433c-4f3a-9708-95e3bdde50ff" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163289    4515 cpu_manager.go:414] "containerMap: removing container" podUID="d05354b1-b462-4af4-af05-80c5a8c0de70" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163293    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d05354b1-b462-4af4-af05-80c5a8c0de70" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163300    4515 cpu_manager.go:414] "containerMap: removing container" podUID="53accdd7-36ef-4487-812b-b0715588261d" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163305    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53accdd7-36ef-4487-812b-b0715588261d" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163315    4515 cpu_manager.go:414] "containerMap: removing container" podUID="228f299b-731d-4375-93a9-fa777801b0a1" containerName="main"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163319    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="228f299b-731d-4375-93a9-fa777801b0a1" containerName="main"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163325    4515 cpu_manager.go:414] "containerMap: removing container" podUID="a471077b-3923-4355-8fda-135f0b8f038a" containerName="main"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163329    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a471077b-3923-4355-8fda-135f0b8f038a" containerName="main"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163335    4515 cpu_manager.go:414] "containerMap: removing container" podUID="b3cb7579-408d-42e0-8ca2-9972bfb54b84" containerName="maas-api"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163340    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3cb7579-408d-42e0-8ca2-9972bfb54b84" containerName="maas-api"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163345    4515 cpu_manager.go:414] "containerMap: removing container" podUID="65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163351    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163359    4515 cpu_manager.go:414] "containerMap: removing container" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163363    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163369    4515 cpu_manager.go:414] "containerMap: removing container" podUID="be97e917-9b14-484e-90f1-b9ad1b678313" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163373    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be97e917-9b14-484e-90f1-b9ad1b678313" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163378    4515 cpu_manager.go:414] "containerMap: removing container" podUID="c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163382    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163390    4515 cpu_manager.go:414] "containerMap: removing container" podUID="7ce83f3e-50cb-44f6-8c6c-4de081152950" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163394    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ce83f3e-50cb-44f6-8c6c-4de081152950" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163439    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3cb7579-408d-42e0-8ca2-9972bfb54b84" containerName="maas-api"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163447    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="65acd2ae-98d9-4492-b3a0-a12f4d0fd8c9" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163454    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="f03b6e9e-52c5-47cd-9de5-2073fc55f0d3" containerName="main"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163461    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="be97e917-9b14-484e-90f1-b9ad1b678313" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163467    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="c70d92a5-6ed0-470d-9c5d-822ffc97bbbc" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163471    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ce83f3e-50cb-44f6-8c6c-4de081152950" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163477    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="73a93fb2-af95-4246-8032-e8ce2025bbdd" containerName="main"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163482    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f990ce6-351c-4a76-87b9-308d20c1b828" containerName="discovery"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163488    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f78c3ae-3bc7-49a5-ad5c-39b6c02d4fbc" containerName="main"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163494    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="de2ef1ec-9dc8-486e-949e-3d12cfdac897" containerName="maas-api"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163499    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc4865ed-8a4f-4432-afc1-61d816b83090" containerName="main"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163503    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="b05cc018-433c-4f3a-9708-95e3bdde50ff" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163509    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="d05354b1-b462-4af4-af05-80c5a8c0de70" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163516    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="53accdd7-36ef-4487-812b-b0715588261d" containerName="istio-proxy"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163521    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="228f299b-731d-4375-93a9-fa777801b0a1" containerName="main"
Nov 02 02:15:00.163647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.163527    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="a471077b-3923-4355-8fda-135f0b8f038a" containerName="main"
Nov 02 02:15:00.167629 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.167609    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9"
Nov 02 02:15:00.219703 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.219675    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnwgr\""
Nov 02 02:15:00.257173 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.257142    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41279ef2-832f-4d10-afa5-229d61e70d51-maas-api-service-ca\") pod \"maas-api-key-cleanup-29893095-zfxt9\" (UID: \"41279ef2-832f-4d10-afa5-229d61e70d51\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9"
Nov 02 02:15:00.257386 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.257336    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv2n8\" (UniqueName: \"kubernetes.io/projected/41279ef2-832f-4d10-afa5-229d61e70d51-kube-api-access-tv2n8\") pod \"maas-api-key-cleanup-29893095-zfxt9\" (UID: \"41279ef2-832f-4d10-afa5-229d61e70d51\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9"
Nov 02 02:15:00.270263 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.270237    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9"]
Nov 02 02:15:00.358540 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.358509    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41279ef2-832f-4d10-afa5-229d61e70d51-maas-api-service-ca\") pod \"maas-api-key-cleanup-29893095-zfxt9\" (UID: \"41279ef2-832f-4d10-afa5-229d61e70d51\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9"
Nov 02 02:15:00.358699 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.358612    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv2n8\" (UniqueName: \"kubernetes.io/projected/41279ef2-832f-4d10-afa5-229d61e70d51-kube-api-access-tv2n8\") pod \"maas-api-key-cleanup-29893095-zfxt9\" (UID: \"41279ef2-832f-4d10-afa5-229d61e70d51\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9"
Nov 02 02:15:00.359146 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.359104    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41279ef2-832f-4d10-afa5-229d61e70d51-maas-api-service-ca\") pod \"maas-api-key-cleanup-29893095-zfxt9\" (UID: \"41279ef2-832f-4d10-afa5-229d61e70d51\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9"
Nov 02 02:15:00.419383 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.419314    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv2n8\" (UniqueName: \"kubernetes.io/projected/41279ef2-832f-4d10-afa5-229d61e70d51-kube-api-access-tv2n8\") pod \"maas-api-key-cleanup-29893095-zfxt9\" (UID: \"41279ef2-832f-4d10-afa5-229d61e70d51\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9"
Nov 02 02:15:00.479547 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.479517    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9"
Nov 02 02:15:00.848135 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:00.848092    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9"]
Nov 02 02:15:00.851615 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:15:00.851582    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41279ef2_832f_4d10_afa5_229d61e70d51.slice/crio-e6ee6ff8af620abf63f66939a7421aec08c1e7a96923be48ad1f039c79ed12ee WatchSource:0}: Error finding container e6ee6ff8af620abf63f66939a7421aec08c1e7a96923be48ad1f039c79ed12ee: Status 404 returned error can't find the container with id e6ee6ff8af620abf63f66939a7421aec08c1e7a96923be48ad1f039c79ed12ee
Nov 02 02:15:01.619473 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:01.619429    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9" event={"ID":"41279ef2-832f-4d10-afa5-229d61e70d51","Type":"ContainerStarted","Data":"28971dde9244cac1da64d1bde7b9c662a9865bf7220c4f77fb5e026762750d6f"}
Nov 02 02:15:01.619980 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:01.619484    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9" event={"ID":"41279ef2-832f-4d10-afa5-229d61e70d51","Type":"ContainerStarted","Data":"e6ee6ff8af620abf63f66939a7421aec08c1e7a96923be48ad1f039c79ed12ee"}
Nov 02 02:15:01.733645 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:01.733578    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9" podStartSLOduration=1.7335623770000002 podStartE2EDuration="1.733562377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:15:01.73342068 +0000 UTC m=+2590.945790502" watchObservedRunningTime="2026-11-02 02:15:01.733562377 +0000 UTC m=+2590.945932185"
Nov 02 02:15:02.624664 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:02.624630    4515 generic.go:356] "Generic (PLEG): container finished" podID="41279ef2-832f-4d10-afa5-229d61e70d51" containerID="28971dde9244cac1da64d1bde7b9c662a9865bf7220c4f77fb5e026762750d6f" exitCode=60
Nov 02 02:15:02.625100 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:02.624715    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9" event={"ID":"41279ef2-832f-4d10-afa5-229d61e70d51","Type":"ContainerDied","Data":"28971dde9244cac1da64d1bde7b9c662a9865bf7220c4f77fb5e026762750d6f"}
Nov 02 02:15:02.625100 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:02.624966    4515 scope.go:122] "RemoveContainer" containerID="28971dde9244cac1da64d1bde7b9c662a9865bf7220c4f77fb5e026762750d6f"
Nov 02 02:15:03.637389 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:03.637306    4515 generic.go:356] "Generic (PLEG): container finished" podID="41279ef2-832f-4d10-afa5-229d61e70d51" containerID="12dffbb0a9b6503305428d419f2cde1ab0e0a99ce7f750298d3152e0ad84ae99" exitCode=60
Nov 02 02:15:03.637738 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:03.637390    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9" event={"ID":"41279ef2-832f-4d10-afa5-229d61e70d51","Type":"ContainerDied","Data":"12dffbb0a9b6503305428d419f2cde1ab0e0a99ce7f750298d3152e0ad84ae99"}
Nov 02 02:15:03.637738 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:03.637467    4515 scope.go:122] "RemoveContainer" containerID="28971dde9244cac1da64d1bde7b9c662a9865bf7220c4f77fb5e026762750d6f"
Nov 02 02:15:03.637738 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:03.637684    4515 scope.go:122] "RemoveContainer" containerID="12dffbb0a9b6503305428d419f2cde1ab0e0a99ce7f750298d3152e0ad84ae99"
Nov 02 02:15:03.637952 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:15:03.637929    4515 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29893095-zfxt9_odh-ai-gateway-infra(41279ef2-832f-4d10-afa5-229d61e70d51)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9" podUID="41279ef2-832f-4d10-afa5-229d61e70d51"
Nov 02 02:15:04.643637 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:04.643612    4515 scope.go:122] "RemoveContainer" containerID="12dffbb0a9b6503305428d419f2cde1ab0e0a99ce7f750298d3152e0ad84ae99"
Nov 02 02:15:04.644000 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:15:04.643827    4515 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29893095-zfxt9_odh-ai-gateway-infra(41279ef2-832f-4d10-afa5-229d61e70d51)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9" podUID="41279ef2-832f-4d10-afa5-229d61e70d51"
Nov 02 02:15:09.958341 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:15:09.958298    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:15:17.946945 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:17.946915    4515 scope.go:122] "RemoveContainer" containerID="12dffbb0a9b6503305428d419f2cde1ab0e0a99ce7f750298d3152e0ad84ae99"
Nov 02 02:15:18.704675 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:18.704631    4515 generic.go:356] "Generic (PLEG): container finished" podID="41279ef2-832f-4d10-afa5-229d61e70d51" containerID="8de17908ca1e2bf1c369c49297597a700441724a0627078549f29405a60e1150" exitCode=60
Nov 02 02:15:18.704861 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:18.704786    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9" event={"ID":"41279ef2-832f-4d10-afa5-229d61e70d51","Type":"ContainerDied","Data":"8de17908ca1e2bf1c369c49297597a700441724a0627078549f29405a60e1150"}
Nov 02 02:15:18.704922 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:18.704880    4515 scope.go:122] "RemoveContainer" containerID="12dffbb0a9b6503305428d419f2cde1ab0e0a99ce7f750298d3152e0ad84ae99"
Nov 02 02:15:18.705439 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:18.705417    4515 scope.go:122] "RemoveContainer" containerID="8de17908ca1e2bf1c369c49297597a700441724a0627078549f29405a60e1150"
Nov 02 02:15:18.705836 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:15:18.705798    4515 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29893095-zfxt9_odh-ai-gateway-infra(41279ef2-832f-4d10-afa5-229d61e70d51)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9" podUID="41279ef2-832f-4d10-afa5-229d61e70d51"
Nov 02 02:15:19.780496 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:19.780465    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"]
Nov 02 02:15:19.780957 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:19.780661    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl" podUID="0ee9e5c7-80ad-4744-9100-e7b9b5c50078" containerName="istio-proxy" containerID="cri-o://fff9efa4ad4658e344fa3df144b82394446dd4b6b6bbd9552430bc152a888b3d" gracePeriod=30
Nov 02 02:15:19.827322 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:19.827285    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9"]
Nov 02 02:15:19.955567 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:19.955544    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9"
Nov 02 02:15:20.151772 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:20.151696    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41279ef2-832f-4d10-afa5-229d61e70d51-kube-api-access-tv2n8\" (UniqueName: \"kubernetes.io/projected/41279ef2-832f-4d10-afa5-229d61e70d51-kube-api-access-tv2n8\") pod \"41279ef2-832f-4d10-afa5-229d61e70d51\" (UID: \"41279ef2-832f-4d10-afa5-229d61e70d51\") "
Nov 02 02:15:20.151910 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:20.151794    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/41279ef2-832f-4d10-afa5-229d61e70d51-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41279ef2-832f-4d10-afa5-229d61e70d51-maas-api-service-ca\") pod \"41279ef2-832f-4d10-afa5-229d61e70d51\" (UID: \"41279ef2-832f-4d10-afa5-229d61e70d51\") "
Nov 02 02:15:20.152188 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:20.152158    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41279ef2-832f-4d10-afa5-229d61e70d51-maas-api-service-ca" pod "41279ef2-832f-4d10-afa5-229d61e70d51" (UID: "41279ef2-832f-4d10-afa5-229d61e70d51"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:15:20.153858 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:20.153835    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41279ef2-832f-4d10-afa5-229d61e70d51-kube-api-access-tv2n8" pod "41279ef2-832f-4d10-afa5-229d61e70d51" (UID: "41279ef2-832f-4d10-afa5-229d61e70d51"). InnerVolumeSpecName "kube-api-access-tv2n8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:15:20.253705 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:20.253676    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41279ef2-832f-4d10-afa5-229d61e70d51-maas-api-service-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:15:20.253705 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:20.253706    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tv2n8\" (UniqueName: \"kubernetes.io/projected/41279ef2-832f-4d10-afa5-229d61e70d51-kube-api-access-tv2n8\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:15:20.716392 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:20.716355    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9" event={"ID":"41279ef2-832f-4d10-afa5-229d61e70d51","Type":"ContainerDied","Data":"e6ee6ff8af620abf63f66939a7421aec08c1e7a96923be48ad1f039c79ed12ee"}
Nov 02 02:15:20.716392 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:20.716380    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9"
Nov 02 02:15:20.716619 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:20.716420    4515 scope.go:122] "RemoveContainer" containerID="8de17908ca1e2bf1c369c49297597a700441724a0627078549f29405a60e1150"
Nov 02 02:15:20.878203 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:20.878166    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9"]
Nov 02 02:15:20.927714 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:20.927683    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893095-zfxt9"]
Nov 02 02:15:20.953151 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:20.953104    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="41279ef2-832f-4d10-afa5-229d61e70d51" path="/var/lib/kubelet/pods/41279ef2-832f-4d10-afa5-229d61e70d51/volumes"
Nov 02 02:15:22.377248 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.377205    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"]
Nov 02 02:15:22.377724 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.377412    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4" podUID="8c81aedb-70f7-4846-b97b-ead08257737a" containerName="maas-api" containerID="cri-o://c43799e8ad388c34c693b35e6d5bf000f2b449de5cf86b0ee9b9936f8b8404f1" gracePeriod=30
Nov 02 02:15:22.640664 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.640602    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 02:15:22.672177 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.672149    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c81aedb-70f7-4846-b97b-ead08257737a-kube-api-access-lcr6x\" (UniqueName: \"kubernetes.io/projected/8c81aedb-70f7-4846-b97b-ead08257737a-kube-api-access-lcr6x\") pod \"8c81aedb-70f7-4846-b97b-ead08257737a\" (UID: \"8c81aedb-70f7-4846-b97b-ead08257737a\") "
Nov 02 02:15:22.672349 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.672196    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c81aedb-70f7-4846-b97b-ead08257737a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c81aedb-70f7-4846-b97b-ead08257737a-maas-api-tls\") pod \"8c81aedb-70f7-4846-b97b-ead08257737a\" (UID: \"8c81aedb-70f7-4846-b97b-ead08257737a\") "
Nov 02 02:15:22.672349 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.672243    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c81aedb-70f7-4846-b97b-ead08257737a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c81aedb-70f7-4846-b97b-ead08257737a-openshift-service-ca\") pod \"8c81aedb-70f7-4846-b97b-ead08257737a\" (UID: \"8c81aedb-70f7-4846-b97b-ead08257737a\") "
Nov 02 02:15:22.672349 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.672317    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c81aedb-70f7-4846-b97b-ead08257737a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c81aedb-70f7-4846-b97b-ead08257737a-metrics-certs\") pod \"8c81aedb-70f7-4846-b97b-ead08257737a\" (UID: \"8c81aedb-70f7-4846-b97b-ead08257737a\") "
Nov 02 02:15:22.672745 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.672715    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c81aedb-70f7-4846-b97b-ead08257737a-openshift-service-ca" pod "8c81aedb-70f7-4846-b97b-ead08257737a" (UID: "8c81aedb-70f7-4846-b97b-ead08257737a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:15:22.674645 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.674611    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c81aedb-70f7-4846-b97b-ead08257737a-metrics-certs" pod "8c81aedb-70f7-4846-b97b-ead08257737a" (UID: "8c81aedb-70f7-4846-b97b-ead08257737a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:15:22.674862 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.674834    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c81aedb-70f7-4846-b97b-ead08257737a-maas-api-tls" pod "8c81aedb-70f7-4846-b97b-ead08257737a" (UID: "8c81aedb-70f7-4846-b97b-ead08257737a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:15:22.674980 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.674917    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c81aedb-70f7-4846-b97b-ead08257737a-kube-api-access-lcr6x" pod "8c81aedb-70f7-4846-b97b-ead08257737a" (UID: "8c81aedb-70f7-4846-b97b-ead08257737a"). InnerVolumeSpecName "kube-api-access-lcr6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:15:22.727026 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.726996    4515 generic.go:356] "Generic (PLEG): container finished" podID="8c81aedb-70f7-4846-b97b-ead08257737a" containerID="c43799e8ad388c34c693b35e6d5bf000f2b449de5cf86b0ee9b9936f8b8404f1" exitCode=0
Nov 02 02:15:22.727187 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.727041    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"
Nov 02 02:15:22.727187 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.727064    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4" event={"ID":"8c81aedb-70f7-4846-b97b-ead08257737a","Type":"ContainerDied","Data":"c43799e8ad388c34c693b35e6d5bf000f2b449de5cf86b0ee9b9936f8b8404f1"}
Nov 02 02:15:22.727187 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.727149    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4" event={"ID":"8c81aedb-70f7-4846-b97b-ead08257737a","Type":"ContainerDied","Data":"bd1c1d57f940cc07dbb97487080f552c4bc0a0bf3e396030a73bd8f60d0b31d4"}
Nov 02 02:15:22.727187 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.727172    4515 scope.go:122] "RemoveContainer" containerID="c43799e8ad388c34c693b35e6d5bf000f2b449de5cf86b0ee9b9936f8b8404f1"
Nov 02 02:15:22.736030 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.736013    4515 scope.go:122] "RemoveContainer" containerID="c43799e8ad388c34c693b35e6d5bf000f2b449de5cf86b0ee9b9936f8b8404f1"
Nov 02 02:15:22.736307 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:15:22.736287    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c43799e8ad388c34c693b35e6d5bf000f2b449de5cf86b0ee9b9936f8b8404f1\": container with ID starting with c43799e8ad388c34c693b35e6d5bf000f2b449de5cf86b0ee9b9936f8b8404f1 not found: ID does not exist" containerID="c43799e8ad388c34c693b35e6d5bf000f2b449de5cf86b0ee9b9936f8b8404f1"
Nov 02 02:15:22.736420 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.736315    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c43799e8ad388c34c693b35e6d5bf000f2b449de5cf86b0ee9b9936f8b8404f1"} err="failed to get container status \"c43799e8ad388c34c693b35e6d5bf000f2b449de5cf86b0ee9b9936f8b8404f1\": rpc error: code = NotFound desc = could not find container \"c43799e8ad388c34c693b35e6d5bf000f2b449de5cf86b0ee9b9936f8b8404f1\": container with ID starting with c43799e8ad388c34c693b35e6d5bf000f2b449de5cf86b0ee9b9936f8b8404f1 not found: ID does not exist"
Nov 02 02:15:22.773315 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.773262    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcr6x\" (UniqueName: \"kubernetes.io/projected/8c81aedb-70f7-4846-b97b-ead08257737a-kube-api-access-lcr6x\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:15:22.773315 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.773295    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c81aedb-70f7-4846-b97b-ead08257737a-maas-api-tls\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:15:22.773315 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.773310    4515 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c81aedb-70f7-4846-b97b-ead08257737a-openshift-service-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:15:22.773315 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.773322    4515 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c81aedb-70f7-4846-b97b-ead08257737a-metrics-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:15:22.870433 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.870392    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"]
Nov 02 02:15:22.875665 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.875635    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c62671-df78d4c8-bsmv4"]
Nov 02 02:15:22.951633 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:22.951605    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c81aedb-70f7-4846-b97b-ead08257737a" path="/var/lib/kubelet/pods/8c81aedb-70f7-4846-b97b-ead08257737a/volumes"
Nov 02 02:15:24.621523 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:24.621483    4515 patch_prober.go:29] interesting pod/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 02:15:24.621919 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:24.621559    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl" podUID="0ee9e5c7-80ad-4744-9100-e7b9b5c50078" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 02:15:25.025629 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.025606    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 02:15:25.095041 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.095009    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istiod-ca-cert\") pod \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") "
Nov 02 02:15:25.095205 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.095050    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-workload-socket\") pod \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") "
Nov 02 02:15:25.095205 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.095129    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-kube-api-access-cgs2m\" (UniqueName: \"kubernetes.io/projected/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-kube-api-access-cgs2m\") pod \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") "
Nov 02 02:15:25.095205 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.095176    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-envoy\") pod \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") "
Nov 02 02:15:25.095384 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.095206    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-credential-socket\") pod \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") "
Nov 02 02:15:25.095384 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.095247    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-token\" (UniqueName: \"kubernetes.io/projected/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-token\") pod \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") "
Nov 02 02:15:25.095384 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.095282    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-podinfo\") pod \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") "
Nov 02 02:15:25.095547 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.095403    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-workload-socket" pod "0ee9e5c7-80ad-4744-9100-e7b9b5c50078" (UID: "0ee9e5c7-80ad-4744-9100-e7b9b5c50078"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:15:25.095547 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.095405    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-workload-certs\") pod \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") "
Nov 02 02:15:25.095547 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.095423    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istiod-ca-cert" pod "0ee9e5c7-80ad-4744-9100-e7b9b5c50078" (UID: "0ee9e5c7-80ad-4744-9100-e7b9b5c50078"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:15:25.095547 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.095490    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-data\") pod \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\" (UID: \"0ee9e5c7-80ad-4744-9100-e7b9b5c50078\") "
Nov 02 02:15:25.095761 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.095731    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-credential-socket" pod "0ee9e5c7-80ad-4744-9100-e7b9b5c50078" (UID: "0ee9e5c7-80ad-4744-9100-e7b9b5c50078"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:15:25.095841 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.095821    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-workload-certs" pod "0ee9e5c7-80ad-4744-9100-e7b9b5c50078" (UID: "0ee9e5c7-80ad-4744-9100-e7b9b5c50078"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:15:25.095902 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.095836    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-data" pod "0ee9e5c7-80ad-4744-9100-e7b9b5c50078" (UID: "0ee9e5c7-80ad-4744-9100-e7b9b5c50078"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:15:25.095902 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.095876    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:15:25.095902 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.095890    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:15:25.095902 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.095899    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:15:25.097438 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.097412    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-podinfo" pod "0ee9e5c7-80ad-4744-9100-e7b9b5c50078" (UID: "0ee9e5c7-80ad-4744-9100-e7b9b5c50078"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:15:25.097536 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.097440    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-kube-api-access-cgs2m" pod "0ee9e5c7-80ad-4744-9100-e7b9b5c50078" (UID: "0ee9e5c7-80ad-4744-9100-e7b9b5c50078"). InnerVolumeSpecName "kube-api-access-cgs2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:15:25.097787 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.097760    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-envoy" pod "0ee9e5c7-80ad-4744-9100-e7b9b5c50078" (UID: "0ee9e5c7-80ad-4744-9100-e7b9b5c50078"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:15:25.097787 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.097774    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-token" pod "0ee9e5c7-80ad-4744-9100-e7b9b5c50078" (UID: "0ee9e5c7-80ad-4744-9100-e7b9b5c50078"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:15:25.196578 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.196548    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgs2m\" (UniqueName: \"kubernetes.io/projected/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-kube-api-access-cgs2m\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:15:25.196578 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.196580    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:15:25.196793 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.196596    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:15:25.196793 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.196609    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:15:25.196793 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.196621    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:15:25.196793 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.196643    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ee9e5c7-80ad-4744-9100-e7b9b5c50078-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:15:25.741725 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.741692    4515 generic.go:356] "Generic (PLEG): container finished" podID="0ee9e5c7-80ad-4744-9100-e7b9b5c50078" containerID="fff9efa4ad4658e344fa3df144b82394446dd4b6b6bbd9552430bc152a888b3d" exitCode=0
Nov 02 02:15:25.742140 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.741744    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"
Nov 02 02:15:25.742140 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.741779    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl" event={"ID":"0ee9e5c7-80ad-4744-9100-e7b9b5c50078","Type":"ContainerDied","Data":"fff9efa4ad4658e344fa3df144b82394446dd4b6b6bbd9552430bc152a888b3d"}
Nov 02 02:15:25.742140 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.741839    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl" event={"ID":"0ee9e5c7-80ad-4744-9100-e7b9b5c50078","Type":"ContainerDied","Data":"c35e018c60b10fe6f9e67ef77777bf1fe345c0535d47688e25da20d30d95bd03"}
Nov 02 02:15:25.742140 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.741862    4515 scope.go:122] "RemoveContainer" containerID="fff9efa4ad4658e344fa3df144b82394446dd4b6b6bbd9552430bc152a888b3d"
Nov 02 02:15:25.751032 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.751017    4515 scope.go:122] "RemoveContainer" containerID="2db996fe421cdea65bfd6940be99ab269c2e614470ae8ecd1a898899afaa4ff4"
Nov 02 02:15:25.758320 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.758302    4515 scope.go:122] "RemoveContainer" containerID="fff9efa4ad4658e344fa3df144b82394446dd4b6b6bbd9552430bc152a888b3d"
Nov 02 02:15:25.758560 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:15:25.758543    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fff9efa4ad4658e344fa3df144b82394446dd4b6b6bbd9552430bc152a888b3d\": container with ID starting with fff9efa4ad4658e344fa3df144b82394446dd4b6b6bbd9552430bc152a888b3d not found: ID does not exist" containerID="fff9efa4ad4658e344fa3df144b82394446dd4b6b6bbd9552430bc152a888b3d"
Nov 02 02:15:25.758607 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.758570    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fff9efa4ad4658e344fa3df144b82394446dd4b6b6bbd9552430bc152a888b3d"} err="failed to get container status \"fff9efa4ad4658e344fa3df144b82394446dd4b6b6bbd9552430bc152a888b3d\": rpc error: code = NotFound desc = could not find container \"fff9efa4ad4658e344fa3df144b82394446dd4b6b6bbd9552430bc152a888b3d\": container with ID starting with fff9efa4ad4658e344fa3df144b82394446dd4b6b6bbd9552430bc152a888b3d not found: ID does not exist"
Nov 02 02:15:25.758607 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.758589    4515 scope.go:122] "RemoveContainer" containerID="2db996fe421cdea65bfd6940be99ab269c2e614470ae8ecd1a898899afaa4ff4"
Nov 02 02:15:25.758829 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:15:25.758811    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2db996fe421cdea65bfd6940be99ab269c2e614470ae8ecd1a898899afaa4ff4\": container with ID starting with 2db996fe421cdea65bfd6940be99ab269c2e614470ae8ecd1a898899afaa4ff4 not found: ID does not exist" containerID="2db996fe421cdea65bfd6940be99ab269c2e614470ae8ecd1a898899afaa4ff4"
Nov 02 02:15:25.758879 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.758835    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2db996fe421cdea65bfd6940be99ab269c2e614470ae8ecd1a898899afaa4ff4"} err="failed to get container status \"2db996fe421cdea65bfd6940be99ab269c2e614470ae8ecd1a898899afaa4ff4\": rpc error: code = NotFound desc = could not find container \"2db996fe421cdea65bfd6940be99ab269c2e614470ae8ecd1a898899afaa4ff4\": container with ID starting with 2db996fe421cdea65bfd6940be99ab269c2e614470ae8ecd1a898899afaa4ff4 not found: ID does not exist"
Nov 02 02:15:25.906107 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.906075    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"]
Nov 02 02:15:25.955689 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:25.955657    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c62671-openshift-default-7659bdbf95-7f5pl"]
Nov 02 02:15:26.951944 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:15:26.951911    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ee9e5c7-80ad-4744-9100-e7b9b5c50078" path="/var/lib/kubelet/pods/0ee9e5c7-80ad-4744-9100-e7b9b5c50078/volumes"
Nov 02 02:15:39.958512 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:15:39.958469    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:16:09.957824 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:16:09.957749    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:16:39.742503 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:39.742464    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"]
Nov 02 02:16:39.743017 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:39.742710    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh" podUID="7bcde5e3-f9bf-4724-8a21-c6a63c13df32" containerName="istio-proxy" containerID="cri-o://c7cc62ba795694a107e63bf8c8e7cc193c09b44a10ff4cb3b768e818ad3de856" gracePeriod=30
Nov 02 02:16:39.957683 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:16:39.957645    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:16:44.993923 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:44.993898    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:16:45.066866 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.066781    4515 generic.go:356] "Generic (PLEG): container finished" podID="7bcde5e3-f9bf-4724-8a21-c6a63c13df32" containerID="c7cc62ba795694a107e63bf8c8e7cc193c09b44a10ff4cb3b768e818ad3de856" exitCode=0
Nov 02 02:16:45.066866 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.066826    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh" event={"ID":"7bcde5e3-f9bf-4724-8a21-c6a63c13df32","Type":"ContainerDied","Data":"c7cc62ba795694a107e63bf8c8e7cc193c09b44a10ff4cb3b768e818ad3de856"}
Nov 02 02:16:45.066866 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.066843    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"
Nov 02 02:16:45.067098 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.066869    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh" event={"ID":"7bcde5e3-f9bf-4724-8a21-c6a63c13df32","Type":"ContainerDied","Data":"e09a933d76d232242df3a76cfe337298e2ca7c18e047811e392cdec8f3f56853"}
Nov 02 02:16:45.067098 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.066891    4515 scope.go:122] "RemoveContainer" containerID="c7cc62ba795694a107e63bf8c8e7cc193c09b44a10ff4cb3b768e818ad3de856"
Nov 02 02:16:45.075222 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.075090    4515 scope.go:122] "RemoveContainer" containerID="8a17a1bc19f12cee6a79978f19371b61dfa0d865648117fe4c5f2390ddff86e5"
Nov 02 02:16:45.083150 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.083131    4515 scope.go:122] "RemoveContainer" containerID="c7cc62ba795694a107e63bf8c8e7cc193c09b44a10ff4cb3b768e818ad3de856"
Nov 02 02:16:45.083423 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:16:45.083401    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7cc62ba795694a107e63bf8c8e7cc193c09b44a10ff4cb3b768e818ad3de856\": container with ID starting with c7cc62ba795694a107e63bf8c8e7cc193c09b44a10ff4cb3b768e818ad3de856 not found: ID does not exist" containerID="c7cc62ba795694a107e63bf8c8e7cc193c09b44a10ff4cb3b768e818ad3de856"
Nov 02 02:16:45.083472 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.083432    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7cc62ba795694a107e63bf8c8e7cc193c09b44a10ff4cb3b768e818ad3de856"} err="failed to get container status \"c7cc62ba795694a107e63bf8c8e7cc193c09b44a10ff4cb3b768e818ad3de856\": rpc error: code = NotFound desc = could not find container \"c7cc62ba795694a107e63bf8c8e7cc193c09b44a10ff4cb3b768e818ad3de856\": container with ID starting with c7cc62ba795694a107e63bf8c8e7cc193c09b44a10ff4cb3b768e818ad3de856 not found: ID does not exist"
Nov 02 02:16:45.083472 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.083450    4515 scope.go:122] "RemoveContainer" containerID="8a17a1bc19f12cee6a79978f19371b61dfa0d865648117fe4c5f2390ddff86e5"
Nov 02 02:16:45.083677 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:16:45.083660    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a17a1bc19f12cee6a79978f19371b61dfa0d865648117fe4c5f2390ddff86e5\": container with ID starting with 8a17a1bc19f12cee6a79978f19371b61dfa0d865648117fe4c5f2390ddff86e5 not found: ID does not exist" containerID="8a17a1bc19f12cee6a79978f19371b61dfa0d865648117fe4c5f2390ddff86e5"
Nov 02 02:16:45.083721 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.083682    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a17a1bc19f12cee6a79978f19371b61dfa0d865648117fe4c5f2390ddff86e5"} err="failed to get container status \"8a17a1bc19f12cee6a79978f19371b61dfa0d865648117fe4c5f2390ddff86e5\": rpc error: code = NotFound desc = could not find container \"8a17a1bc19f12cee6a79978f19371b61dfa0d865648117fe4c5f2390ddff86e5\": container with ID starting with 8a17a1bc19f12cee6a79978f19371b61dfa0d865648117fe4c5f2390ddff86e5 not found: ID does not exist"
Nov 02 02:16:45.120890 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.120860    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-podinfo\") pod \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") "
Nov 02 02:16:45.121077 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.120922    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istiod-ca-cert\") pod \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") "
Nov 02 02:16:45.121077 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.120956    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-credential-socket\") pod \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") "
Nov 02 02:16:45.121077 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.120989    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-envoy\") pod \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") "
Nov 02 02:16:45.121077 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.121010    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-kube-api-access-lwj45\" (UniqueName: \"kubernetes.io/projected/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-kube-api-access-lwj45\") pod \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") "
Nov 02 02:16:45.121077 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.121048    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-token\" (UniqueName: \"kubernetes.io/projected/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-token\") pod \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") "
Nov 02 02:16:45.121382 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.121109    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-workload-certs\") pod \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") "
Nov 02 02:16:45.121382 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.121163    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-data\") pod \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") "
Nov 02 02:16:45.121382 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.121192    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-workload-socket\") pod \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\" (UID: \"7bcde5e3-f9bf-4724-8a21-c6a63c13df32\") "
Nov 02 02:16:45.121382 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.121348    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-credential-socket" pod "7bcde5e3-f9bf-4724-8a21-c6a63c13df32" (UID: "7bcde5e3-f9bf-4724-8a21-c6a63c13df32"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:16:45.121590 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.121456    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istiod-ca-cert" pod "7bcde5e3-f9bf-4724-8a21-c6a63c13df32" (UID: "7bcde5e3-f9bf-4724-8a21-c6a63c13df32"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:16:45.121590 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.121573    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:16:45.121590 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.121552    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-workload-certs" pod "7bcde5e3-f9bf-4724-8a21-c6a63c13df32" (UID: "7bcde5e3-f9bf-4724-8a21-c6a63c13df32"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:16:45.121748 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.121563    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-workload-socket" pod "7bcde5e3-f9bf-4724-8a21-c6a63c13df32" (UID: "7bcde5e3-f9bf-4724-8a21-c6a63c13df32"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:16:45.121748 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.121625    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-data" pod "7bcde5e3-f9bf-4724-8a21-c6a63c13df32" (UID: "7bcde5e3-f9bf-4724-8a21-c6a63c13df32"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:16:45.123396 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.123371    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-podinfo" pod "7bcde5e3-f9bf-4724-8a21-c6a63c13df32" (UID: "7bcde5e3-f9bf-4724-8a21-c6a63c13df32"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:16:45.123538 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.123502    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-envoy" pod "7bcde5e3-f9bf-4724-8a21-c6a63c13df32" (UID: "7bcde5e3-f9bf-4724-8a21-c6a63c13df32"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:16:45.123821 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.123798    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-kube-api-access-lwj45" pod "7bcde5e3-f9bf-4724-8a21-c6a63c13df32" (UID: "7bcde5e3-f9bf-4724-8a21-c6a63c13df32"). InnerVolumeSpecName "kube-api-access-lwj45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:16:45.123870 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.123816    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-token" pod "7bcde5e3-f9bf-4724-8a21-c6a63c13df32" (UID: "7bcde5e3-f9bf-4724-8a21-c6a63c13df32"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:16:45.222841 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.222807    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:16:45.222841 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.222836    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:16:45.222841 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.222846    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:16:45.223103 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.222855    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:16:45.223103 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.222864    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:16:45.223103 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.222873    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:16:45.223103 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.222881    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:16:45.223103 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.222889    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwj45\" (UniqueName: \"kubernetes.io/projected/7bcde5e3-f9bf-4724-8a21-c6a63c13df32-kube-api-access-lwj45\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:16:45.486825 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.486792    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"]
Nov 02 02:16:45.490994 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:45.490967    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-1c942d-openshift-default-59bc44865-vq8zh"]
Nov 02 02:16:46.951424 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:46.951391    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7bcde5e3-f9bf-4724-8a21-c6a63c13df32" path="/var/lib/kubelet/pods/7bcde5e3-f9bf-4724-8a21-c6a63c13df32/volumes"
Nov 02 02:16:51.231836 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:51.231805    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 02:16:51.238218 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:16:51.238193    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 02:17:09.957556 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:17:09.957518    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:17:16.160583 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:16.160549    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"]
Nov 02 02:17:31.179857 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.179816    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" podUID="6357653d-8952-4aff-b01b-c1e3dc3969ce" containerName="main" containerID="cri-o://865c13aa142fe81b1c8e3d35c4d8a9dfdafe976316e5fa55c47fcdb317272513" gracePeriod=45
Nov 02 02:17:31.424251 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.424227    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 02:17:31.520476 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.520444    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6357653d-8952-4aff-b01b-c1e3dc3969ce-tls-certs\" (UniqueName: \"kubernetes.io/secret/6357653d-8952-4aff-b01b-c1e3dc3969ce-tls-certs\") pod \"6357653d-8952-4aff-b01b-c1e3dc3969ce\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") "
Nov 02 02:17:31.520627 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.520490    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6357653d-8952-4aff-b01b-c1e3dc3969ce-kube-api-access-4p4kw\" (UniqueName: \"kubernetes.io/projected/6357653d-8952-4aff-b01b-c1e3dc3969ce-kube-api-access-4p4kw\") pod \"6357653d-8952-4aff-b01b-c1e3dc3969ce\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") "
Nov 02 02:17:31.520627 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.520533    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-dshm\") pod \"6357653d-8952-4aff-b01b-c1e3dc3969ce\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") "
Nov 02 02:17:31.520627 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.520605    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-tmp-dir\") pod \"6357653d-8952-4aff-b01b-c1e3dc3969ce\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") "
Nov 02 02:17:31.520761 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.520627    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-home\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-home\") pod \"6357653d-8952-4aff-b01b-c1e3dc3969ce\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") "
Nov 02 02:17:31.520761 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.520670    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-model-cache\") pod \"6357653d-8952-4aff-b01b-c1e3dc3969ce\" (UID: \"6357653d-8952-4aff-b01b-c1e3dc3969ce\") "
Nov 02 02:17:31.520986 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.520933    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-tmp-dir" pod "6357653d-8952-4aff-b01b-c1e3dc3969ce" (UID: "6357653d-8952-4aff-b01b-c1e3dc3969ce"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:17:31.520986 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.520977    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-home" pod "6357653d-8952-4aff-b01b-c1e3dc3969ce" (UID: "6357653d-8952-4aff-b01b-c1e3dc3969ce"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:17:31.521176 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.520964    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-model-cache" pod "6357653d-8952-4aff-b01b-c1e3dc3969ce" (UID: "6357653d-8952-4aff-b01b-c1e3dc3969ce"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:17:31.522688 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.522667    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-dshm" pod "6357653d-8952-4aff-b01b-c1e3dc3969ce" (UID: "6357653d-8952-4aff-b01b-c1e3dc3969ce"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:17:31.522780 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.522745    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6357653d-8952-4aff-b01b-c1e3dc3969ce-kube-api-access-4p4kw" pod "6357653d-8952-4aff-b01b-c1e3dc3969ce" (UID: "6357653d-8952-4aff-b01b-c1e3dc3969ce"). InnerVolumeSpecName "kube-api-access-4p4kw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:17:31.522823 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.522784    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6357653d-8952-4aff-b01b-c1e3dc3969ce-tls-certs" pod "6357653d-8952-4aff-b01b-c1e3dc3969ce" (UID: "6357653d-8952-4aff-b01b-c1e3dc3969ce"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:17:31.622410 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.622376    4515 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-tmp-dir\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:17:31.622410 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.622409    4515 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-home\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:17:31.622607 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.622418    4515 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-model-cache\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:17:31.622607 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.622428    4515 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6357653d-8952-4aff-b01b-c1e3dc3969ce-tls-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:17:31.622607 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.622439    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4p4kw\" (UniqueName: \"kubernetes.io/projected/6357653d-8952-4aff-b01b-c1e3dc3969ce-kube-api-access-4p4kw\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:17:31.622607 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:31.622448    4515 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6357653d-8952-4aff-b01b-c1e3dc3969ce-dshm\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:17:32.250012 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:32.249981    4515 generic.go:356] "Generic (PLEG): container finished" podID="6357653d-8952-4aff-b01b-c1e3dc3969ce" containerID="865c13aa142fe81b1c8e3d35c4d8a9dfdafe976316e5fa55c47fcdb317272513" exitCode=0
Nov 02 02:17:32.250469 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:32.250022    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" event={"ID":"6357653d-8952-4aff-b01b-c1e3dc3969ce","Type":"ContainerDied","Data":"865c13aa142fe81b1c8e3d35c4d8a9dfdafe976316e5fa55c47fcdb317272513"}
Nov 02 02:17:32.250469 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:32.250039    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"
Nov 02 02:17:32.250469 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:32.250056    4515 scope.go:122] "RemoveContainer" containerID="865c13aa142fe81b1c8e3d35c4d8a9dfdafe976316e5fa55c47fcdb317272513"
Nov 02 02:17:32.250469 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:32.250046    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n" event={"ID":"6357653d-8952-4aff-b01b-c1e3dc3969ce","Type":"ContainerDied","Data":"44120d471341ba81480d97d1e3cb96c209c519ca5aca47d95d54ddbabfb70c7d"}
Nov 02 02:17:32.258411 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:32.258386    4515 scope.go:122] "RemoveContainer" containerID="865c13aa142fe81b1c8e3d35c4d8a9dfdafe976316e5fa55c47fcdb317272513"
Nov 02 02:17:32.258696 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:17:32.258671    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"865c13aa142fe81b1c8e3d35c4d8a9dfdafe976316e5fa55c47fcdb317272513\": container with ID starting with 865c13aa142fe81b1c8e3d35c4d8a9dfdafe976316e5fa55c47fcdb317272513 not found: ID does not exist" containerID="865c13aa142fe81b1c8e3d35c4d8a9dfdafe976316e5fa55c47fcdb317272513"
Nov 02 02:17:32.258794 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:32.258706    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"865c13aa142fe81b1c8e3d35c4d8a9dfdafe976316e5fa55c47fcdb317272513"} err="failed to get container status \"865c13aa142fe81b1c8e3d35c4d8a9dfdafe976316e5fa55c47fcdb317272513\": rpc error: code = NotFound desc = could not find container \"865c13aa142fe81b1c8e3d35c4d8a9dfdafe976316e5fa55c47fcdb317272513\": container with ID starting with 865c13aa142fe81b1c8e3d35c4d8a9dfdafe976316e5fa55c47fcdb317272513 not found: ID does not exist"
Nov 02 02:17:32.414685 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:32.414652    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"]
Nov 02 02:17:32.464030 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:32.464002    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-0ebdb3/distinct-2-w6-0ebdb3-kserve-6df7c47c86-gzj7n"]
Nov 02 02:17:32.951476 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:32.951395    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6357653d-8952-4aff-b01b-c1e3dc3969ce" path="/var/lib/kubelet/pods/6357653d-8952-4aff-b01b-c1e3dc3969ce/volumes"
Nov 02 02:17:39.957517 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:17:39.957479    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:17:43.003855 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.003821    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"]
Nov 02 02:17:43.004309 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.004016    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw" podUID="1a387928-4b07-4d1b-b906-89d36cfe2f0e" containerName="maas-api" containerID="cri-o://83bd5d3d579e3a7a73ab23bad2120e87b75c1bc9a54c259efdc9e198d79beb9e" gracePeriod=30
Nov 02 02:17:43.266351 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.266285    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 02:17:43.300166 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.300105    4515 generic.go:356] "Generic (PLEG): container finished" podID="1a387928-4b07-4d1b-b906-89d36cfe2f0e" containerID="83bd5d3d579e3a7a73ab23bad2120e87b75c1bc9a54c259efdc9e198d79beb9e" exitCode=0
Nov 02 02:17:43.300166 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.300166    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"
Nov 02 02:17:43.300376 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.300181    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw" event={"ID":"1a387928-4b07-4d1b-b906-89d36cfe2f0e","Type":"ContainerDied","Data":"83bd5d3d579e3a7a73ab23bad2120e87b75c1bc9a54c259efdc9e198d79beb9e"}
Nov 02 02:17:43.300376 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.300240    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw" event={"ID":"1a387928-4b07-4d1b-b906-89d36cfe2f0e","Type":"ContainerDied","Data":"173ae12144019a2f1907637eaad49ad61af1aed1a7ffd99f8bca821a1e9e2e6c"}
Nov 02 02:17:43.300376 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.300265    4515 scope.go:122] "RemoveContainer" containerID="83bd5d3d579e3a7a73ab23bad2120e87b75c1bc9a54c259efdc9e198d79beb9e"
Nov 02 02:17:43.308420 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.308403    4515 scope.go:122] "RemoveContainer" containerID="83bd5d3d579e3a7a73ab23bad2120e87b75c1bc9a54c259efdc9e198d79beb9e"
Nov 02 02:17:43.308676 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:17:43.308659    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83bd5d3d579e3a7a73ab23bad2120e87b75c1bc9a54c259efdc9e198d79beb9e\": container with ID starting with 83bd5d3d579e3a7a73ab23bad2120e87b75c1bc9a54c259efdc9e198d79beb9e not found: ID does not exist" containerID="83bd5d3d579e3a7a73ab23bad2120e87b75c1bc9a54c259efdc9e198d79beb9e"
Nov 02 02:17:43.308725 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.308687    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83bd5d3d579e3a7a73ab23bad2120e87b75c1bc9a54c259efdc9e198d79beb9e"} err="failed to get container status \"83bd5d3d579e3a7a73ab23bad2120e87b75c1bc9a54c259efdc9e198d79beb9e\": rpc error: code = NotFound desc = could not find container \"83bd5d3d579e3a7a73ab23bad2120e87b75c1bc9a54c259efdc9e198d79beb9e\": container with ID starting with 83bd5d3d579e3a7a73ab23bad2120e87b75c1bc9a54c259efdc9e198d79beb9e not found: ID does not exist"
Nov 02 02:17:43.433784 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.433736    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a387928-4b07-4d1b-b906-89d36cfe2f0e-kube-api-access-vn46p\" (UniqueName: \"kubernetes.io/projected/1a387928-4b07-4d1b-b906-89d36cfe2f0e-kube-api-access-vn46p\") pod \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\" (UID: \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\") "
Nov 02 02:17:43.433997 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.433807    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a387928-4b07-4d1b-b906-89d36cfe2f0e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a387928-4b07-4d1b-b906-89d36cfe2f0e-maas-api-tls\") pod \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\" (UID: \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\") "
Nov 02 02:17:43.433997 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.433892    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a387928-4b07-4d1b-b906-89d36cfe2f0e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a387928-4b07-4d1b-b906-89d36cfe2f0e-openshift-service-ca\") pod \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\" (UID: \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\") "
Nov 02 02:17:43.433997 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.433926    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a387928-4b07-4d1b-b906-89d36cfe2f0e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a387928-4b07-4d1b-b906-89d36cfe2f0e-metrics-certs\") pod \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\" (UID: \"1a387928-4b07-4d1b-b906-89d36cfe2f0e\") "
Nov 02 02:17:43.434495 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.434464    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a387928-4b07-4d1b-b906-89d36cfe2f0e-openshift-service-ca" pod "1a387928-4b07-4d1b-b906-89d36cfe2f0e" (UID: "1a387928-4b07-4d1b-b906-89d36cfe2f0e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:17:43.436451 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.436411    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a387928-4b07-4d1b-b906-89d36cfe2f0e-maas-api-tls" pod "1a387928-4b07-4d1b-b906-89d36cfe2f0e" (UID: "1a387928-4b07-4d1b-b906-89d36cfe2f0e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:17:43.437558 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.437524    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a387928-4b07-4d1b-b906-89d36cfe2f0e-metrics-certs" pod "1a387928-4b07-4d1b-b906-89d36cfe2f0e" (UID: "1a387928-4b07-4d1b-b906-89d36cfe2f0e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:17:43.437863 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.437838    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a387928-4b07-4d1b-b906-89d36cfe2f0e-kube-api-access-vn46p" pod "1a387928-4b07-4d1b-b906-89d36cfe2f0e" (UID: "1a387928-4b07-4d1b-b906-89d36cfe2f0e"). InnerVolumeSpecName "kube-api-access-vn46p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:17:43.534678 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.534591    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vn46p\" (UniqueName: \"kubernetes.io/projected/1a387928-4b07-4d1b-b906-89d36cfe2f0e-kube-api-access-vn46p\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:17:43.534678 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.534627    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a387928-4b07-4d1b-b906-89d36cfe2f0e-maas-api-tls\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:17:43.534678 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.534639    4515 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a387928-4b07-4d1b-b906-89d36cfe2f0e-openshift-service-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:17:43.534678 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.534648    4515 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a387928-4b07-4d1b-b906-89d36cfe2f0e-metrics-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:17:43.721536 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.721495    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"]
Nov 02 02:17:43.725595 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:43.725566    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-0ebdb3-6bff98cdbf-rwmbw"]
Nov 02 02:17:44.952162 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:17:44.952130    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a387928-4b07-4d1b-b906-89d36cfe2f0e" path="/var/lib/kubelet/pods/1a387928-4b07-4d1b-b906-89d36cfe2f0e/volumes"
Nov 02 02:18:09.957675 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:18:09.957632    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:18:39.957900 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:18:39.957862    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:19:09.958486 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:19:09.958401    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:19:39.958200 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:19:39.958165    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:20:09.958163 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:20:09.958110    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:20:39.958524 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:20:39.958482    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:21:09.957880 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:21:09.957841    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:21:39.958618 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:21:39.958578    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:21:51.255442 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:21:51.255410    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 02:21:51.261393 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:21:51.261371    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 02:22:09.957820 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:22:09.957737    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:22:39.957682 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:22:39.957642    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:23:09.957681 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:23:09.957636    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:23:40.009995 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:23:40.009956    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:24:09.957964 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:24:09.957926    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:24:39.957725 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:24:39.957689    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:25:09.958482 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:25:09.958393    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:25:39.957386 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:25:39.957348    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:26:09.957722 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:26:09.957685    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:26:13.299251 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299224    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"]
Nov 02 02:26:13.299628 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299505    4515 cpu_manager.go:414] "containerMap: removing container" podUID="41279ef2-832f-4d10-afa5-229d61e70d51" containerName="cleanup"
Nov 02 02:26:13.299628 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299515    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41279ef2-832f-4d10-afa5-229d61e70d51" containerName="cleanup"
Nov 02 02:26:13.299628 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299528    4515 cpu_manager.go:414] "containerMap: removing container" podUID="0ee9e5c7-80ad-4744-9100-e7b9b5c50078" containerName="istio-proxy"
Nov 02 02:26:13.299628 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299533    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ee9e5c7-80ad-4744-9100-e7b9b5c50078" containerName="istio-proxy"
Nov 02 02:26:13.299628 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299545    4515 cpu_manager.go:414] "containerMap: removing container" podUID="6357653d-8952-4aff-b01b-c1e3dc3969ce" containerName="main"
Nov 02 02:26:13.299628 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299551    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6357653d-8952-4aff-b01b-c1e3dc3969ce" containerName="main"
Nov 02 02:26:13.299628 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299558    4515 cpu_manager.go:414] "containerMap: removing container" podUID="41279ef2-832f-4d10-afa5-229d61e70d51" containerName="cleanup"
Nov 02 02:26:13.299628 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299563    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41279ef2-832f-4d10-afa5-229d61e70d51" containerName="cleanup"
Nov 02 02:26:13.299628 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299569    4515 cpu_manager.go:414] "containerMap: removing container" podUID="7bcde5e3-f9bf-4724-8a21-c6a63c13df32" containerName="istio-proxy"
Nov 02 02:26:13.299628 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299574    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7bcde5e3-f9bf-4724-8a21-c6a63c13df32" containerName="istio-proxy"
Nov 02 02:26:13.299628 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299589    4515 cpu_manager.go:414] "containerMap: removing container" podUID="1a387928-4b07-4d1b-b906-89d36cfe2f0e" containerName="maas-api"
Nov 02 02:26:13.299628 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299594    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a387928-4b07-4d1b-b906-89d36cfe2f0e" containerName="maas-api"
Nov 02 02:26:13.299628 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299604    4515 cpu_manager.go:414] "containerMap: removing container" podUID="8c81aedb-70f7-4846-b97b-ead08257737a" containerName="maas-api"
Nov 02 02:26:13.299628 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299609    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c81aedb-70f7-4846-b97b-ead08257737a" containerName="maas-api"
Nov 02 02:26:13.300032 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299654    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a387928-4b07-4d1b-b906-89d36cfe2f0e" containerName="maas-api"
Nov 02 02:26:13.300032 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299663    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="41279ef2-832f-4d10-afa5-229d61e70d51" containerName="cleanup"
Nov 02 02:26:13.300032 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299669    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ee9e5c7-80ad-4744-9100-e7b9b5c50078" containerName="istio-proxy"
Nov 02 02:26:13.300032 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299674    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c81aedb-70f7-4846-b97b-ead08257737a" containerName="maas-api"
Nov 02 02:26:13.300032 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299679    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="41279ef2-832f-4d10-afa5-229d61e70d51" containerName="cleanup"
Nov 02 02:26:13.300032 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299684    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ee9e5c7-80ad-4744-9100-e7b9b5c50078" containerName="istio-proxy"
Nov 02 02:26:13.300032 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299690    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="6357653d-8952-4aff-b01b-c1e3dc3969ce" containerName="main"
Nov 02 02:26:13.300032 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299695    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="7bcde5e3-f9bf-4724-8a21-c6a63c13df32" containerName="istio-proxy"
Nov 02 02:26:13.300032 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.299703    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="7bcde5e3-f9bf-4724-8a21-c6a63c13df32" containerName="istio-proxy"
Nov 02 02:26:13.302529 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.302513    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.354714 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.354688    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-054a89\"/\"default-dockercfg-mck7z\""
Nov 02 02:26:13.382305 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.382272    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-model-cache\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.382460 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.382389    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7531bce5-a620-4792-84a8-e36ebcd07ef7-tls-certs\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.382460 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.382418    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-dshm\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.382578 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.382508    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-home\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.382578 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.382565    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cv98\" (UniqueName: \"kubernetes.io/projected/7531bce5-a620-4792-84a8-e36ebcd07ef7-kube-api-access-4cv98\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.382683 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.382612    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-tmp-dir\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.404052 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.404027    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-054a89\"/\"simulator-single-main-054a89-kserve-self-signed-certs\""
Nov 02 02:26:13.404052 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.404029    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-054a89\"/\"openshift-service-ca.crt\""
Nov 02 02:26:13.404243 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.404078    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-054a89\"/\"kube-root-ca.crt\""
Nov 02 02:26:13.405532 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.405514    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"]
Nov 02 02:26:13.483978 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.483935    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7531bce5-a620-4792-84a8-e36ebcd07ef7-tls-certs\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.483978 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.483982    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-dshm\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.484292 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.484017    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-home\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.484292 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.484053    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cv98\" (UniqueName: \"kubernetes.io/projected/7531bce5-a620-4792-84a8-e36ebcd07ef7-kube-api-access-4cv98\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.484292 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.484089    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-tmp-dir\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.484292 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.484172    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-model-cache\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.484531 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.484508    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-home\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.484575 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.484526    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-tmp-dir\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.484635 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.484614    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-model-cache\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.486386 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.486358    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-dshm\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.486573 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.486556    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7531bce5-a620-4792-84a8-e36ebcd07ef7-tls-certs\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.544621 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.544596    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cv98\" (UniqueName: \"kubernetes.io/projected/7531bce5-a620-4792-84a8-e36ebcd07ef7-kube-api-access-4cv98\") pod \"simulator-single-main-054a89-kserve-54bb47876f-s7rf6\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") " pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.613687 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.613597    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:13.988076 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:13.988039    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"]
Nov 02 02:26:13.991383 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:26:13.991357    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7531bce5_a620_4792_84a8_e36ebcd07ef7.slice/crio-c5a9b561736d7cdab6c5dd55546828c6923daae381b07e791403163184ec654c WatchSource:0}: Error finding container c5a9b561736d7cdab6c5dd55546828c6923daae381b07e791403163184ec654c: Status 404 returned error can't find the container with id c5a9b561736d7cdab6c5dd55546828c6923daae381b07e791403163184ec654c
Nov 02 02:26:14.415595 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:14.415499    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6" event={"ID":"7531bce5-a620-4792-84a8-e36ebcd07ef7","Type":"ContainerStarted","Data":"9bc197b876c83c8ec4833baccdad60cb8630f44b96664ba8dacbc9c5d50a8db9"}
Nov 02 02:26:14.415595 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:14.415542    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6" event={"ID":"7531bce5-a620-4792-84a8-e36ebcd07ef7","Type":"ContainerStarted","Data":"c5a9b561736d7cdab6c5dd55546828c6923daae381b07e791403163184ec654c"}
Nov 02 02:26:14.531790 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:14.531726    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6" podStartSLOduration=1.5317083569999999 podStartE2EDuration="1.531708357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:26:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:26:14.531085479 +0000 UTC m=+3263.743455287" watchObservedRunningTime="2026-11-02 02:26:14.531708357 +0000 UTC m=+3263.744078167"
Nov 02 02:26:14.614537 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:14.614506    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:23.614703 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:23.614663    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:23.627194 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:23.627164    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:24.463187 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:24.463157    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:26:39.957458 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:26:39.957419    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:26:51.280083 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:51.280051    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 02:26:51.286037 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:26:51.286015    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 02:27:09.958474 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:27:09.958436    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:27:15.019569 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:15.019525    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"]
Nov 02 02:27:30.040750 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.040704    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6" podUID="7531bce5-a620-4792-84a8-e36ebcd07ef7" containerName="main" containerID="cri-o://9bc197b876c83c8ec4833baccdad60cb8630f44b96664ba8dacbc9c5d50a8db9" gracePeriod=45
Nov 02 02:27:30.278193 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.278164    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:27:30.349669 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.349585    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7531bce5-a620-4792-84a8-e36ebcd07ef7-tls-certs\" (UniqueName: \"kubernetes.io/secret/7531bce5-a620-4792-84a8-e36ebcd07ef7-tls-certs\") pod \"7531bce5-a620-4792-84a8-e36ebcd07ef7\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") "
Nov 02 02:27:30.349669 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.349636    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-model-cache\") pod \"7531bce5-a620-4792-84a8-e36ebcd07ef7\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") "
Nov 02 02:27:30.349669 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.349659    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-tmp-dir\") pod \"7531bce5-a620-4792-84a8-e36ebcd07ef7\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") "
Nov 02 02:27:30.349977 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.349715    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-dshm\") pod \"7531bce5-a620-4792-84a8-e36ebcd07ef7\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") "
Nov 02 02:27:30.349977 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.349735    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7531bce5-a620-4792-84a8-e36ebcd07ef7-kube-api-access-4cv98\" (UniqueName: \"kubernetes.io/projected/7531bce5-a620-4792-84a8-e36ebcd07ef7-kube-api-access-4cv98\") pod \"7531bce5-a620-4792-84a8-e36ebcd07ef7\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") "
Nov 02 02:27:30.349977 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.349763    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-home\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-home\") pod \"7531bce5-a620-4792-84a8-e36ebcd07ef7\" (UID: \"7531bce5-a620-4792-84a8-e36ebcd07ef7\") "
Nov 02 02:27:30.349977 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.349914    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-model-cache" pod "7531bce5-a620-4792-84a8-e36ebcd07ef7" (UID: "7531bce5-a620-4792-84a8-e36ebcd07ef7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:27:30.349977 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.349958    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-tmp-dir" pod "7531bce5-a620-4792-84a8-e36ebcd07ef7" (UID: "7531bce5-a620-4792-84a8-e36ebcd07ef7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:27:30.350297 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.350093    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-home" pod "7531bce5-a620-4792-84a8-e36ebcd07ef7" (UID: "7531bce5-a620-4792-84a8-e36ebcd07ef7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:27:30.351945 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.351911    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7531bce5-a620-4792-84a8-e36ebcd07ef7-tls-certs" pod "7531bce5-a620-4792-84a8-e36ebcd07ef7" (UID: "7531bce5-a620-4792-84a8-e36ebcd07ef7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:27:30.351945 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.351938    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-dshm" pod "7531bce5-a620-4792-84a8-e36ebcd07ef7" (UID: "7531bce5-a620-4792-84a8-e36ebcd07ef7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:27:30.352083 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.351949    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7531bce5-a620-4792-84a8-e36ebcd07ef7-kube-api-access-4cv98" pod "7531bce5-a620-4792-84a8-e36ebcd07ef7" (UID: "7531bce5-a620-4792-84a8-e36ebcd07ef7"). InnerVolumeSpecName "kube-api-access-4cv98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:27:30.451192 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.451154    4515 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-model-cache\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:27:30.451192 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.451190    4515 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-tmp-dir\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:27:30.451441 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.451206    4515 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-dshm\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:27:30.451441 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.451216    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cv98\" (UniqueName: \"kubernetes.io/projected/7531bce5-a620-4792-84a8-e36ebcd07ef7-kube-api-access-4cv98\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:27:30.451441 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.451226    4515 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7531bce5-a620-4792-84a8-e36ebcd07ef7-home\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:27:30.451441 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.451235    4515 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7531bce5-a620-4792-84a8-e36ebcd07ef7-tls-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:27:30.713639 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.713603    4515 generic.go:356] "Generic (PLEG): container finished" podID="7531bce5-a620-4792-84a8-e36ebcd07ef7" containerID="9bc197b876c83c8ec4833baccdad60cb8630f44b96664ba8dacbc9c5d50a8db9" exitCode=0
Nov 02 02:27:30.713813 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.713644    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"
Nov 02 02:27:30.713813 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.713673    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6" event={"ID":"7531bce5-a620-4792-84a8-e36ebcd07ef7","Type":"ContainerDied","Data":"9bc197b876c83c8ec4833baccdad60cb8630f44b96664ba8dacbc9c5d50a8db9"}
Nov 02 02:27:30.713813 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.713711    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6" event={"ID":"7531bce5-a620-4792-84a8-e36ebcd07ef7","Type":"ContainerDied","Data":"c5a9b561736d7cdab6c5dd55546828c6923daae381b07e791403163184ec654c"}
Nov 02 02:27:30.713813 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.713730    4515 scope.go:122] "RemoveContainer" containerID="9bc197b876c83c8ec4833baccdad60cb8630f44b96664ba8dacbc9c5d50a8db9"
Nov 02 02:27:30.722062 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.722045    4515 scope.go:122] "RemoveContainer" containerID="9bc197b876c83c8ec4833baccdad60cb8630f44b96664ba8dacbc9c5d50a8db9"
Nov 02 02:27:30.722469 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:27:30.722437    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bc197b876c83c8ec4833baccdad60cb8630f44b96664ba8dacbc9c5d50a8db9\": container with ID starting with 9bc197b876c83c8ec4833baccdad60cb8630f44b96664ba8dacbc9c5d50a8db9 not found: ID does not exist" containerID="9bc197b876c83c8ec4833baccdad60cb8630f44b96664ba8dacbc9c5d50a8db9"
Nov 02 02:27:30.722513 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.722477    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bc197b876c83c8ec4833baccdad60cb8630f44b96664ba8dacbc9c5d50a8db9"} err="failed to get container status \"9bc197b876c83c8ec4833baccdad60cb8630f44b96664ba8dacbc9c5d50a8db9\": rpc error: code = NotFound desc = could not find container \"9bc197b876c83c8ec4833baccdad60cb8630f44b96664ba8dacbc9c5d50a8db9\": container with ID starting with 9bc197b876c83c8ec4833baccdad60cb8630f44b96664ba8dacbc9c5d50a8db9 not found: ID does not exist"
Nov 02 02:27:30.876900 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.876867    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"]
Nov 02 02:27:30.926555 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.926519    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-054a89/simulator-single-main-054a89-kserve-54bb47876f-s7rf6"]
Nov 02 02:27:30.951231 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:27:30.951198    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7531bce5-a620-4792-84a8-e36ebcd07ef7" path="/var/lib/kubelet/pods/7531bce5-a620-4792-84a8-e36ebcd07ef7/volumes"
Nov 02 02:27:39.958794 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:27:39.958754    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:28:09.957777 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:28:09.957693    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:28:39.957904 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:28:39.957865    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:29:09.958203 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:29:09.958158    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:29:39.958021 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:29:39.957984    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:30:09.958301 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:30:09.958260    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:30:39.958354 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:30:39.958310    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:30:46.925545 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.925510    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"]
Nov 02 02:30:46.926011 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.925804    4515 cpu_manager.go:414] "containerMap: removing container" podUID="0ee9e5c7-80ad-4744-9100-e7b9b5c50078" containerName="istio-proxy"
Nov 02 02:30:46.926011 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.925814    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ee9e5c7-80ad-4744-9100-e7b9b5c50078" containerName="istio-proxy"
Nov 02 02:30:46.926011 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.925830    4515 cpu_manager.go:414] "containerMap: removing container" podUID="41279ef2-832f-4d10-afa5-229d61e70d51" containerName="cleanup"
Nov 02 02:30:46.926011 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.925835    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41279ef2-832f-4d10-afa5-229d61e70d51" containerName="cleanup"
Nov 02 02:30:46.926011 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.925842    4515 cpu_manager.go:414] "containerMap: removing container" podUID="7bcde5e3-f9bf-4724-8a21-c6a63c13df32" containerName="istio-proxy"
Nov 02 02:30:46.926011 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.925850    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7bcde5e3-f9bf-4724-8a21-c6a63c13df32" containerName="istio-proxy"
Nov 02 02:30:46.926011 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.925859    4515 cpu_manager.go:414] "containerMap: removing container" podUID="7531bce5-a620-4792-84a8-e36ebcd07ef7" containerName="main"
Nov 02 02:30:46.926011 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.925864    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7531bce5-a620-4792-84a8-e36ebcd07ef7" containerName="main"
Nov 02 02:30:46.926011 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.925910    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="41279ef2-832f-4d10-afa5-229d61e70d51" containerName="cleanup"
Nov 02 02:30:46.926011 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.925920    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="7531bce5-a620-4792-84a8-e36ebcd07ef7" containerName="main"
Nov 02 02:30:46.928718 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.928703    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:46.981799 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.981764    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-18df32\"/\"openshift-service-ca.crt\""
Nov 02 02:30:46.990170 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.990135    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-home\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:46.990334 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.990225    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzwrk\" (UniqueName: \"kubernetes.io/projected/1628e905-db5f-4cca-8140-d554f5e8c70f-kube-api-access-hzwrk\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:46.990334 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.990284    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-dshm\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:46.990334 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.990308    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-tmp-dir\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:46.990493 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.990372    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-model-cache\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:46.990493 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:46.990436    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1628e905-db5f-4cca-8140-d554f5e8c70f-tls-certs\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:47.031744 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.031713    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-18df32\"/\"kube-root-ca.crt\""
Nov 02 02:30:47.031904 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.031755    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-18df32\"/\"simulator-single-main-18df32-kserve-self-signed-certs\""
Nov 02 02:30:47.031904 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.031813    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-18df32\"/\"default-dockercfg-k5qx7\""
Nov 02 02:30:47.033319 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.033301    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"]
Nov 02 02:30:47.091574 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.091538    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-model-cache\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:47.091787 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.091599    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1628e905-db5f-4cca-8140-d554f5e8c70f-tls-certs\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:47.091787 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.091650    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-home\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:47.091787 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.091697    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzwrk\" (UniqueName: \"kubernetes.io/projected/1628e905-db5f-4cca-8140-d554f5e8c70f-kube-api-access-hzwrk\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:47.091787 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.091723    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-dshm\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:47.091787 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.091744    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-tmp-dir\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:47.092061 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.092039    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-model-cache\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:47.092141 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.092071    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-home\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:47.092141 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.092109    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-tmp-dir\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:47.093982 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.093959    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-dshm\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:47.094224 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.094205    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1628e905-db5f-4cca-8140-d554f5e8c70f-tls-certs\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:47.152610 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.152573    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzwrk\" (UniqueName: \"kubernetes.io/projected/1628e905-db5f-4cca-8140-d554f5e8c70f-kube-api-access-hzwrk\") pod \"simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") " pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:47.239640 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.239600    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:47.614578 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:47.614508    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"]
Nov 02 02:30:47.617769 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:30:47.617741    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1628e905_db5f_4cca_8140_d554f5e8c70f.slice/crio-96b4a58477cfdc69909da44e43af23cee380d0d1cd582380cf19f4e51a97a8e8 WatchSource:0}: Error finding container 96b4a58477cfdc69909da44e43af23cee380d0d1cd582380cf19f4e51a97a8e8: Status 404 returned error can't find the container with id 96b4a58477cfdc69909da44e43af23cee380d0d1cd582380cf19f4e51a97a8e8
Nov 02 02:30:48.510424 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:48.510390    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4" event={"ID":"1628e905-db5f-4cca-8140-d554f5e8c70f","Type":"ContainerStarted","Data":"a65250a10da7a36c22a08cbede3df4f294fb864e07d1ad56ee7e21141705b9bb"}
Nov 02 02:30:48.510424 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:48.510430    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4" event={"ID":"1628e905-db5f-4cca-8140-d554f5e8c70f","Type":"ContainerStarted","Data":"96b4a58477cfdc69909da44e43af23cee380d0d1cd582380cf19f4e51a97a8e8"}
Nov 02 02:30:48.628880 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:48.628823    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4" podStartSLOduration=2.628805674 podStartE2EDuration="2.628805674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:30:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:30:48.627420505 +0000 UTC m=+3537.839790313" watchObservedRunningTime="2026-11-02 02:30:48.628805674 +0000 UTC m=+3537.841175483"
Nov 02 02:30:49.240588 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:49.240555    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:57.240486 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:57.240449    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:57.253747 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:57.253721    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:30:57.560732 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:30:57.560656    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:31:09.958640 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:31:09.958602    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:31:39.958578 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:31:39.958538    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:31:48.755627 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:31:48.755587    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"]
Nov 02 02:31:51.303765 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:31:51.303734    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 02:31:51.309751 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:31:51.309730    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 02:32:03.774716 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:03.774658    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4" podUID="1628e905-db5f-4cca-8140-d554f5e8c70f" containerName="main" containerID="cri-o://a65250a10da7a36c22a08cbede3df4f294fb864e07d1ad56ee7e21141705b9bb" gracePeriod=45
Nov 02 02:32:04.014493 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.014467    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:32:04.073387 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.073320    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1628e905-db5f-4cca-8140-d554f5e8c70f-tls-certs\" (UniqueName: \"kubernetes.io/secret/1628e905-db5f-4cca-8140-d554f5e8c70f-tls-certs\") pod \"1628e905-db5f-4cca-8140-d554f5e8c70f\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") "
Nov 02 02:32:04.073387 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.073357    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-model-cache\") pod \"1628e905-db5f-4cca-8140-d554f5e8c70f\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") "
Nov 02 02:32:04.073565 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.073392    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-home\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-home\") pod \"1628e905-db5f-4cca-8140-d554f5e8c70f\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") "
Nov 02 02:32:04.073605 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.073587    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1628e905-db5f-4cca-8140-d554f5e8c70f-kube-api-access-hzwrk\" (UniqueName: \"kubernetes.io/projected/1628e905-db5f-4cca-8140-d554f5e8c70f-kube-api-access-hzwrk\") pod \"1628e905-db5f-4cca-8140-d554f5e8c70f\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") "
Nov 02 02:32:04.073649 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.073622    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-model-cache" pod "1628e905-db5f-4cca-8140-d554f5e8c70f" (UID: "1628e905-db5f-4cca-8140-d554f5e8c70f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:32:04.073649 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.073639    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-home" pod "1628e905-db5f-4cca-8140-d554f5e8c70f" (UID: "1628e905-db5f-4cca-8140-d554f5e8c70f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:32:04.073733 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.073687    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-tmp-dir\") pod \"1628e905-db5f-4cca-8140-d554f5e8c70f\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") "
Nov 02 02:32:04.073733 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.073720    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-dshm\") pod \"1628e905-db5f-4cca-8140-d554f5e8c70f\" (UID: \"1628e905-db5f-4cca-8140-d554f5e8c70f\") "
Nov 02 02:32:04.073931 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.073909    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-tmp-dir" pod "1628e905-db5f-4cca-8140-d554f5e8c70f" (UID: "1628e905-db5f-4cca-8140-d554f5e8c70f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:32:04.074023 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.073998    4515 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-model-cache\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:32:04.074023 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.074014    4515 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-home\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:32:04.074107 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.074050    4515 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-tmp-dir\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:32:04.075661 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.075633    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-dshm" pod "1628e905-db5f-4cca-8140-d554f5e8c70f" (UID: "1628e905-db5f-4cca-8140-d554f5e8c70f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:32:04.075759 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.075683    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1628e905-db5f-4cca-8140-d554f5e8c70f-tls-certs" pod "1628e905-db5f-4cca-8140-d554f5e8c70f" (UID: "1628e905-db5f-4cca-8140-d554f5e8c70f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:32:04.075759 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.075710    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1628e905-db5f-4cca-8140-d554f5e8c70f-kube-api-access-hzwrk" pod "1628e905-db5f-4cca-8140-d554f5e8c70f" (UID: "1628e905-db5f-4cca-8140-d554f5e8c70f"). InnerVolumeSpecName "kube-api-access-hzwrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:32:04.174782 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.174752    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzwrk\" (UniqueName: \"kubernetes.io/projected/1628e905-db5f-4cca-8140-d554f5e8c70f-kube-api-access-hzwrk\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:32:04.174782 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.174779    4515 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1628e905-db5f-4cca-8140-d554f5e8c70f-dshm\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:32:04.174986 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.174790    4515 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1628e905-db5f-4cca-8140-d554f5e8c70f-tls-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:32:04.804703 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.804667    4515 generic.go:356] "Generic (PLEG): container finished" podID="1628e905-db5f-4cca-8140-d554f5e8c70f" containerID="a65250a10da7a36c22a08cbede3df4f294fb864e07d1ad56ee7e21141705b9bb" exitCode=0
Nov 02 02:32:04.805145 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.804716    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"
Nov 02 02:32:04.805145 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.804743    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4" event={"ID":"1628e905-db5f-4cca-8140-d554f5e8c70f","Type":"ContainerDied","Data":"a65250a10da7a36c22a08cbede3df4f294fb864e07d1ad56ee7e21141705b9bb"}
Nov 02 02:32:04.805145 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.804786    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4" event={"ID":"1628e905-db5f-4cca-8140-d554f5e8c70f","Type":"ContainerDied","Data":"96b4a58477cfdc69909da44e43af23cee380d0d1cd582380cf19f4e51a97a8e8"}
Nov 02 02:32:04.805145 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.804803    4515 scope.go:122] "RemoveContainer" containerID="a65250a10da7a36c22a08cbede3df4f294fb864e07d1ad56ee7e21141705b9bb"
Nov 02 02:32:04.813388 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.813370    4515 scope.go:122] "RemoveContainer" containerID="a65250a10da7a36c22a08cbede3df4f294fb864e07d1ad56ee7e21141705b9bb"
Nov 02 02:32:04.813645 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:32:04.813630    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a65250a10da7a36c22a08cbede3df4f294fb864e07d1ad56ee7e21141705b9bb\": container with ID starting with a65250a10da7a36c22a08cbede3df4f294fb864e07d1ad56ee7e21141705b9bb not found: ID does not exist" containerID="a65250a10da7a36c22a08cbede3df4f294fb864e07d1ad56ee7e21141705b9bb"
Nov 02 02:32:04.813688 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.813653    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a65250a10da7a36c22a08cbede3df4f294fb864e07d1ad56ee7e21141705b9bb"} err="failed to get container status \"a65250a10da7a36c22a08cbede3df4f294fb864e07d1ad56ee7e21141705b9bb\": rpc error: code = NotFound desc = could not find container \"a65250a10da7a36c22a08cbede3df4f294fb864e07d1ad56ee7e21141705b9bb\": container with ID starting with a65250a10da7a36c22a08cbede3df4f294fb864e07d1ad56ee7e21141705b9bb not found: ID does not exist"
Nov 02 02:32:04.967886 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:04.967857    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"]
Nov 02 02:32:05.016822 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:05.016785    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-18df32/simulator-single-main-18df32-kserve-7bcc4fbd-ttpp4"]
Nov 02 02:32:06.951828 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:06.951796    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1628e905-db5f-4cca-8140-d554f5e8c70f" path="/var/lib/kubelet/pods/1628e905-db5f-4cca-8140-d554f5e8c70f/volumes"
Nov 02 02:32:09.957641 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:32:09.957605    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:32:30.423504 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.423468    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"]
Nov 02 02:32:30.423994 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.423793    4515 cpu_manager.go:414] "containerMap: removing container" podUID="1628e905-db5f-4cca-8140-d554f5e8c70f" containerName="main"
Nov 02 02:32:30.423994 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.423804    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1628e905-db5f-4cca-8140-d554f5e8c70f" containerName="main"
Nov 02 02:32:30.423994 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.423877    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="1628e905-db5f-4cca-8140-d554f5e8c70f" containerName="main"
Nov 02 02:32:30.426748 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.426726    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.504763 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.504726    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wmgn\" (UniqueName: \"kubernetes.io/projected/795be554-1e9e-4290-a45f-8bd4560e5ad8-kube-api-access-7wmgn\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.504934 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.504769    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-tmp-dir\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.504934 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.504873    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-home\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.505040 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.504955    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-dshm\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.505040 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.505019    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-model-cache\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.505142 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.505054    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/795be554-1e9e-4290-a45f-8bd4560e5ad8-tls-certs\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.528755 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.528734    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-2b8b1197-kserve-self-signed-certs\""
Nov 02 02:32:30.530274 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.530253    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"]
Nov 02 02:32:30.605917 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.605882    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-home\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.605917 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.605922    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-dshm\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.606226 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.605943    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-model-cache\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.606226 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.605974    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/795be554-1e9e-4290-a45f-8bd4560e5ad8-tls-certs\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.606226 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.605994    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wmgn\" (UniqueName: \"kubernetes.io/projected/795be554-1e9e-4290-a45f-8bd4560e5ad8-kube-api-access-7wmgn\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.606226 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.606016    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-tmp-dir\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.606439 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.606424    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-home\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.606512 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.606487    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-model-cache\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.606564 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.606493    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-tmp-dir\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.608269 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.608246    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-dshm\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.608465 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.608448    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/795be554-1e9e-4290-a45f-8bd4560e5ad8-tls-certs\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.666409 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.666383    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wmgn\" (UniqueName: \"kubernetes.io/projected/795be554-1e9e-4290-a45f-8bd4560e5ad8-kube-api-access-7wmgn\") pod \"e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") " pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:30.737417 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:30.737381    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:31.110765 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:31.110606    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"]
Nov 02 02:32:31.113360 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:32:31.113330    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod795be554_1e9e_4290_a45f_8bd4560e5ad8.slice/crio-ef5109fcdd9910facdf248bffd41957d0c421f9493b2e0e0ea8faec4b6159f91 WatchSource:0}: Error finding container ef5109fcdd9910facdf248bffd41957d0c421f9493b2e0e0ea8faec4b6159f91: Status 404 returned error can't find the container with id ef5109fcdd9910facdf248bffd41957d0c421f9493b2e0e0ea8faec4b6159f91
Nov 02 02:32:31.912231 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:31.912194    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk" event={"ID":"795be554-1e9e-4290-a45f-8bd4560e5ad8","Type":"ContainerStarted","Data":"9c69bd86d10649edc07aa3aa5f524317b623696a434fcec483c9524cc0b3d9ba"}
Nov 02 02:32:31.912719 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:31.912243    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk" event={"ID":"795be554-1e9e-4290-a45f-8bd4560e5ad8","Type":"ContainerStarted","Data":"ef5109fcdd9910facdf248bffd41957d0c421f9493b2e0e0ea8faec4b6159f91"}
Nov 02 02:32:32.029402 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:32.029348    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk" podStartSLOduration=2.02933046 podStartE2EDuration="2.02933046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:32:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:32:32.028233598 +0000 UTC m=+3641.240603420" watchObservedRunningTime="2026-11-02 02:32:32.02933046 +0000 UTC m=+3641.241700262"
Nov 02 02:32:32.737634 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:32.737606    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:37.985768 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:37.985248    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-7wmnn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 02:32:37.985768 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:37.985317    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-7wmnn" podUID="dd24d725-bfe3-4cde-9fc7-5d6d4e497995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 02:32:39.957943 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:32:39.957906    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:32:40.737960 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:40.737923    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:40.750332 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:40.750309    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:40.978319 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:40.978287    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:32:56.485750 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.485708    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"]
Nov 02 02:32:56.488935 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.488913    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.567367 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.567337    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-model-cache\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.567367 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.567379    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxrkb\" (UniqueName: \"kubernetes.io/projected/cd61c5ee-a5e5-4536-8103-a07153e0de16-kube-api-access-hxrkb\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.567568 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.567465    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd61c5ee-a5e5-4536-8103-a07153e0de16-tls-certs\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.567568 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.567541    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-tmp-dir\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.567568 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.567562    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-dshm\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.567679 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.567580    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-home\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.590612 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.590580    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-2b8b1197-kserve-self-signed-certs\""
Nov 02 02:32:56.591896 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.591873    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"]
Nov 02 02:32:56.668428 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.668391    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-home\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.668575 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.668443    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-model-cache\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.668575 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.668562    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxrkb\" (UniqueName: \"kubernetes.io/projected/cd61c5ee-a5e5-4536-8103-a07153e0de16-kube-api-access-hxrkb\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.668681 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.668629    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd61c5ee-a5e5-4536-8103-a07153e0de16-tls-certs\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.668744 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.668728    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-tmp-dir\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.668801 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.668761    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-dshm\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.668801 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.668777    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-model-cache\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.668914 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.668848    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-home\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.675594 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.669563    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-tmp-dir\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.675594 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.672296    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd61c5ee-a5e5-4536-8103-a07153e0de16-tls-certs\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.676139 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.676094    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-dshm\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.728820 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.728789    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxrkb\" (UniqueName: \"kubernetes.io/projected/cd61c5ee-a5e5-4536-8103-a07153e0de16-kube-api-access-hxrkb\") pod \"e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") " pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:56.799661 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:56.799576    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:32:57.174791 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:57.174712    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"]
Nov 02 02:32:57.178187 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:32:57.178145    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd61c5ee_a5e5_4536_8103_a07153e0de16.slice/crio-9e76d2e867bd0a92d48767a19f2d9cb88159d0769480b5d1e8d1f739dee4afe8 WatchSource:0}: Error finding container 9e76d2e867bd0a92d48767a19f2d9cb88159d0769480b5d1e8d1f739dee4afe8: Status 404 returned error can't find the container with id 9e76d2e867bd0a92d48767a19f2d9cb88159d0769480b5d1e8d1f739dee4afe8
Nov 02 02:32:58.034244 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:58.034213    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk" event={"ID":"cd61c5ee-a5e5-4536-8103-a07153e0de16","Type":"ContainerStarted","Data":"d661d59cdc163518b0dbd84c41cd9081cc27e808981bae5c7f10b661baba0328"}
Nov 02 02:32:58.034244 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:58.034250    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk" event={"ID":"cd61c5ee-a5e5-4536-8103-a07153e0de16","Type":"ContainerStarted","Data":"9e76d2e867bd0a92d48767a19f2d9cb88159d0769480b5d1e8d1f739dee4afe8"}
Nov 02 02:32:58.150623 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:58.150563    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk" podStartSLOduration=2.150549027 podStartE2EDuration="2.150549027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:32:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:32:58.150413202 +0000 UTC m=+3667.362783011" watchObservedRunningTime="2026-11-02 02:32:58.150549027 +0000 UTC m=+3667.362918832"
Nov 02 02:32:58.800207 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:32:58.800171    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:33:06.799949 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:06.799908    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:33:06.812404 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:06.812373    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:33:07.106577 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:07.106488    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:33:09.958364 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:33:09.958326    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:33:30.513286 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:30.513240    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-gnmf7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 02:33:30.513818 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:30.513310    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7" podUID="612f51d4-bab6-4b28-b837-094c15c5166f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 02:33:38.446526 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:38.446493    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"]
Nov 02 02:33:39.958733 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:33:39.958689    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:33:45.106143 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:45.085476    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"]
Nov 02 02:33:45.517284 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:45.517243    4515 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-gnmf7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 02:33:45.517475 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:45.517307    4515 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-gnmf7" podUID="612f51d4-bab6-4b28-b837-094c15c5166f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 02:33:53.468012 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.467946    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk" podUID="795be554-1e9e-4290-a45f-8bd4560e5ad8" containerName="main" containerID="cri-o://9c69bd86d10649edc07aa3aa5f524317b623696a434fcec483c9524cc0b3d9ba" gracePeriod=45
Nov 02 02:33:53.712960 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.712933    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:33:53.814442 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.814365    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/795be554-1e9e-4290-a45f-8bd4560e5ad8-kube-api-access-7wmgn\" (UniqueName: \"kubernetes.io/projected/795be554-1e9e-4290-a45f-8bd4560e5ad8-kube-api-access-7wmgn\") pod \"795be554-1e9e-4290-a45f-8bd4560e5ad8\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") "
Nov 02 02:33:53.814442 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.814412    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-dshm\") pod \"795be554-1e9e-4290-a45f-8bd4560e5ad8\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") "
Nov 02 02:33:53.814627 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.814488    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/795be554-1e9e-4290-a45f-8bd4560e5ad8-tls-certs\" (UniqueName: \"kubernetes.io/secret/795be554-1e9e-4290-a45f-8bd4560e5ad8-tls-certs\") pod \"795be554-1e9e-4290-a45f-8bd4560e5ad8\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") "
Nov 02 02:33:53.814627 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.814526    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-tmp-dir\") pod \"795be554-1e9e-4290-a45f-8bd4560e5ad8\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") "
Nov 02 02:33:53.814627 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.814552    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-model-cache\") pod \"795be554-1e9e-4290-a45f-8bd4560e5ad8\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") "
Nov 02 02:33:53.814627 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.814589    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-home\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-home\") pod \"795be554-1e9e-4290-a45f-8bd4560e5ad8\" (UID: \"795be554-1e9e-4290-a45f-8bd4560e5ad8\") "
Nov 02 02:33:53.814929 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.814864    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-tmp-dir" pod "795be554-1e9e-4290-a45f-8bd4560e5ad8" (UID: "795be554-1e9e-4290-a45f-8bd4560e5ad8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:33:53.814929 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.814873    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-model-cache" pod "795be554-1e9e-4290-a45f-8bd4560e5ad8" (UID: "795be554-1e9e-4290-a45f-8bd4560e5ad8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:33:53.815211 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.814952    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-home" pod "795be554-1e9e-4290-a45f-8bd4560e5ad8" (UID: "795be554-1e9e-4290-a45f-8bd4560e5ad8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:33:53.816669 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.816636    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-dshm" pod "795be554-1e9e-4290-a45f-8bd4560e5ad8" (UID: "795be554-1e9e-4290-a45f-8bd4560e5ad8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:33:53.816767 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.816720    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/795be554-1e9e-4290-a45f-8bd4560e5ad8-kube-api-access-7wmgn" pod "795be554-1e9e-4290-a45f-8bd4560e5ad8" (UID: "795be554-1e9e-4290-a45f-8bd4560e5ad8"). InnerVolumeSpecName "kube-api-access-7wmgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:33:53.816807 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.816765    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/795be554-1e9e-4290-a45f-8bd4560e5ad8-tls-certs" pod "795be554-1e9e-4290-a45f-8bd4560e5ad8" (UID: "795be554-1e9e-4290-a45f-8bd4560e5ad8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:33:53.916257 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.916219    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wmgn\" (UniqueName: \"kubernetes.io/projected/795be554-1e9e-4290-a45f-8bd4560e5ad8-kube-api-access-7wmgn\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:33:53.916257 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.916250    4515 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-dshm\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:33:53.916257 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.916261    4515 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/795be554-1e9e-4290-a45f-8bd4560e5ad8-tls-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:33:53.916504 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.916289    4515 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-tmp-dir\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:33:53.916504 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.916300    4515 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-model-cache\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:33:53.916504 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:53.916309    4515 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/795be554-1e9e-4290-a45f-8bd4560e5ad8-home\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:33:54.355756 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:54.355719    4515 generic.go:356] "Generic (PLEG): container finished" podID="795be554-1e9e-4290-a45f-8bd4560e5ad8" containerID="9c69bd86d10649edc07aa3aa5f524317b623696a434fcec483c9524cc0b3d9ba" exitCode=0
Nov 02 02:33:54.355962 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:54.355771    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"
Nov 02 02:33:54.355962 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:54.355800    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk" event={"ID":"795be554-1e9e-4290-a45f-8bd4560e5ad8","Type":"ContainerDied","Data":"9c69bd86d10649edc07aa3aa5f524317b623696a434fcec483c9524cc0b3d9ba"}
Nov 02 02:33:54.355962 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:54.355843    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk" event={"ID":"795be554-1e9e-4290-a45f-8bd4560e5ad8","Type":"ContainerDied","Data":"ef5109fcdd9910facdf248bffd41957d0c421f9493b2e0e0ea8faec4b6159f91"}
Nov 02 02:33:54.355962 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:54.355860    4515 scope.go:122] "RemoveContainer" containerID="9c69bd86d10649edc07aa3aa5f524317b623696a434fcec483c9524cc0b3d9ba"
Nov 02 02:33:54.365044 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:54.365024    4515 scope.go:122] "RemoveContainer" containerID="9c69bd86d10649edc07aa3aa5f524317b623696a434fcec483c9524cc0b3d9ba"
Nov 02 02:33:54.365370 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:33:54.365348    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c69bd86d10649edc07aa3aa5f524317b623696a434fcec483c9524cc0b3d9ba\": container with ID starting with 9c69bd86d10649edc07aa3aa5f524317b623696a434fcec483c9524cc0b3d9ba not found: ID does not exist" containerID="9c69bd86d10649edc07aa3aa5f524317b623696a434fcec483c9524cc0b3d9ba"
Nov 02 02:33:54.365412 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:54.365379    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c69bd86d10649edc07aa3aa5f524317b623696a434fcec483c9524cc0b3d9ba"} err="failed to get container status \"9c69bd86d10649edc07aa3aa5f524317b623696a434fcec483c9524cc0b3d9ba\": rpc error: code = NotFound desc = could not find container \"9c69bd86d10649edc07aa3aa5f524317b623696a434fcec483c9524cc0b3d9ba\": container with ID starting with 9c69bd86d10649edc07aa3aa5f524317b623696a434fcec483c9524cc0b3d9ba not found: ID does not exist"
Nov 02 02:33:54.519202 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:54.519167    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"]
Nov 02 02:33:54.523514 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:54.523488    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-2b8b1197-kserve-597899f4b5-dsjmk"]
Nov 02 02:33:54.952056 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:33:54.952023    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="795be554-1e9e-4290-a45f-8bd4560e5ad8" path="/var/lib/kubelet/pods/795be554-1e9e-4290-a45f-8bd4560e5ad8/volumes"
Nov 02 02:34:00.214341 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.214304    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk" podUID="cd61c5ee-a5e5-4536-8103-a07153e0de16" containerName="main" containerID="cri-o://d661d59cdc163518b0dbd84c41cd9081cc27e808981bae5c7f10b661baba0328" gracePeriod=45
Nov 02 02:34:00.383771 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.383744    4515 generic.go:356] "Generic (PLEG): container finished" podID="cd61c5ee-a5e5-4536-8103-a07153e0de16" containerID="d661d59cdc163518b0dbd84c41cd9081cc27e808981bae5c7f10b661baba0328" exitCode=0
Nov 02 02:34:00.383901 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.383818    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk" event={"ID":"cd61c5ee-a5e5-4536-8103-a07153e0de16","Type":"ContainerDied","Data":"d661d59cdc163518b0dbd84c41cd9081cc27e808981bae5c7f10b661baba0328"}
Nov 02 02:34:00.461867 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.461841    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:34:00.578892 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.578409    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-model-cache\") pod \"cd61c5ee-a5e5-4536-8103-a07153e0de16\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") "
Nov 02 02:34:00.578892 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.578481    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-tmp-dir\") pod \"cd61c5ee-a5e5-4536-8103-a07153e0de16\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") "
Nov 02 02:34:00.578892 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.578509    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-dshm\") pod \"cd61c5ee-a5e5-4536-8103-a07153e0de16\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") "
Nov 02 02:34:00.578892 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.578546    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-home\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-home\") pod \"cd61c5ee-a5e5-4536-8103-a07153e0de16\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") "
Nov 02 02:34:00.578892 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.578623    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cd61c5ee-a5e5-4536-8103-a07153e0de16-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd61c5ee-a5e5-4536-8103-a07153e0de16-tls-certs\") pod \"cd61c5ee-a5e5-4536-8103-a07153e0de16\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") "
Nov 02 02:34:00.578892 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.578712    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd61c5ee-a5e5-4536-8103-a07153e0de16-kube-api-access-hxrkb\" (UniqueName: \"kubernetes.io/projected/cd61c5ee-a5e5-4536-8103-a07153e0de16-kube-api-access-hxrkb\") pod \"cd61c5ee-a5e5-4536-8103-a07153e0de16\" (UID: \"cd61c5ee-a5e5-4536-8103-a07153e0de16\") "
Nov 02 02:34:00.578892 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.578763    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-model-cache" pod "cd61c5ee-a5e5-4536-8103-a07153e0de16" (UID: "cd61c5ee-a5e5-4536-8103-a07153e0de16"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:34:00.579395 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.578994    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-home" pod "cd61c5ee-a5e5-4536-8103-a07153e0de16" (UID: "cd61c5ee-a5e5-4536-8103-a07153e0de16"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:34:00.579395 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.579173    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-tmp-dir" pod "cd61c5ee-a5e5-4536-8103-a07153e0de16" (UID: "cd61c5ee-a5e5-4536-8103-a07153e0de16"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:34:00.579840 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.579751    4515 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-tmp-dir\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:34:00.579840 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.579779    4515 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-home\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:34:00.579840 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.579792    4515 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-model-cache\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:34:00.581442 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.581406    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd61c5ee-a5e5-4536-8103-a07153e0de16-kube-api-access-hxrkb" pod "cd61c5ee-a5e5-4536-8103-a07153e0de16" (UID: "cd61c5ee-a5e5-4536-8103-a07153e0de16"). InnerVolumeSpecName "kube-api-access-hxrkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:34:00.581548 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.581475    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd61c5ee-a5e5-4536-8103-a07153e0de16-tls-certs" pod "cd61c5ee-a5e5-4536-8103-a07153e0de16" (UID: "cd61c5ee-a5e5-4536-8103-a07153e0de16"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:34:00.581548 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.581483    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-dshm" pod "cd61c5ee-a5e5-4536-8103-a07153e0de16" (UID: "cd61c5ee-a5e5-4536-8103-a07153e0de16"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:34:00.680634 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.680603    4515 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd61c5ee-a5e5-4536-8103-a07153e0de16-dshm\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:34:00.680634 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.680631    4515 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd61c5ee-a5e5-4536-8103-a07153e0de16-tls-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:34:00.680634 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:00.680643    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxrkb\" (UniqueName: \"kubernetes.io/projected/cd61c5ee-a5e5-4536-8103-a07153e0de16-kube-api-access-hxrkb\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:34:01.389831 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:01.389754    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"
Nov 02 02:34:01.389831 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:01.389765    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk" event={"ID":"cd61c5ee-a5e5-4536-8103-a07153e0de16","Type":"ContainerDied","Data":"9e76d2e867bd0a92d48767a19f2d9cb88159d0769480b5d1e8d1f739dee4afe8"}
Nov 02 02:34:01.389831 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:01.389809    4515 scope.go:122] "RemoveContainer" containerID="d661d59cdc163518b0dbd84c41cd9081cc27e808981bae5c7f10b661baba0328"
Nov 02 02:34:01.552538 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:01.552503    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"]
Nov 02 02:34:01.602306 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:01.602272    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-2b8b1197-kserve-df9c577b5-f2rjk"]
Nov 02 02:34:02.955580 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:02.955505    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd61c5ee-a5e5-4536-8103-a07153e0de16" path="/var/lib/kubelet/pods/cd61c5ee-a5e5-4536-8103-a07153e0de16/volumes"
Nov 02 02:34:10.015625 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:34:10.015589    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:34:39.958532 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:34:39.958492    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:34:46.144567 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:46.144524    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"]
Nov 02 02:34:46.145259 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:46.144740    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q" podUID="20040f1c-ca52-4c4c-af8f-fbe2ade458e4" containerName="maas-api" containerID="cri-o://4c892fddc2576f6e5fd94895fcc21962381da0903f38057f96a5882125f2543d" gracePeriod=30
Nov 02 02:34:47.392950 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.392926    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 02:34:47.413842 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.413816    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-openshift-service-ca\") pod \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\" (UID: \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\") "
Nov 02 02:34:47.413949 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.413855    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-maas-api-tls\") pod \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\" (UID: \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\") "
Nov 02 02:34:47.413949 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.413888    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-kube-api-access-c5bs8\" (UniqueName: \"kubernetes.io/projected/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-kube-api-access-c5bs8\") pod \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\" (UID: \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\") "
Nov 02 02:34:47.413949 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.413940    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-metrics-certs\") pod \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\" (UID: \"20040f1c-ca52-4c4c-af8f-fbe2ade458e4\") "
Nov 02 02:34:47.414192 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.414163    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-openshift-service-ca" pod "20040f1c-ca52-4c4c-af8f-fbe2ade458e4" (UID: "20040f1c-ca52-4c4c-af8f-fbe2ade458e4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:34:47.416013 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.415983    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-kube-api-access-c5bs8" pod "20040f1c-ca52-4c4c-af8f-fbe2ade458e4" (UID: "20040f1c-ca52-4c4c-af8f-fbe2ade458e4"). InnerVolumeSpecName "kube-api-access-c5bs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:34:47.416013 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.416006    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-maas-api-tls" pod "20040f1c-ca52-4c4c-af8f-fbe2ade458e4" (UID: "20040f1c-ca52-4c4c-af8f-fbe2ade458e4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:34:47.416157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.416022    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-metrics-certs" pod "20040f1c-ca52-4c4c-af8f-fbe2ade458e4" (UID: "20040f1c-ca52-4c4c-af8f-fbe2ade458e4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:34:47.516236 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.516144    4515 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-openshift-service-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:34:47.516236 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.516179    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-maas-api-tls\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:34:47.516236 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.516191    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5bs8\" (UniqueName: \"kubernetes.io/projected/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-kube-api-access-c5bs8\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:34:47.516236 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.516204    4515 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20040f1c-ca52-4c4c-af8f-fbe2ade458e4-metrics-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:34:47.602581 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.602541    4515 generic.go:356] "Generic (PLEG): container finished" podID="20040f1c-ca52-4c4c-af8f-fbe2ade458e4" containerID="4c892fddc2576f6e5fd94895fcc21962381da0903f38057f96a5882125f2543d" exitCode=0
Nov 02 02:34:47.602751 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.602595    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"
Nov 02 02:34:47.602751 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.602609    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q" event={"ID":"20040f1c-ca52-4c4c-af8f-fbe2ade458e4","Type":"ContainerDied","Data":"4c892fddc2576f6e5fd94895fcc21962381da0903f38057f96a5882125f2543d"}
Nov 02 02:34:47.602751 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.602662    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66b669b4fd-j675q" event={"ID":"20040f1c-ca52-4c4c-af8f-fbe2ade458e4","Type":"ContainerDied","Data":"e19f0098461dd48619f6a64dd5063556101cc245ed24ef0fbf5435b0e469d4fa"}
Nov 02 02:34:47.602751 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.602691    4515 scope.go:122] "RemoveContainer" containerID="4c892fddc2576f6e5fd94895fcc21962381da0903f38057f96a5882125f2543d"
Nov 02 02:34:47.611786 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.611763    4515 scope.go:122] "RemoveContainer" containerID="4c892fddc2576f6e5fd94895fcc21962381da0903f38057f96a5882125f2543d"
Nov 02 02:34:47.612075 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:34:47.612053    4515 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c892fddc2576f6e5fd94895fcc21962381da0903f38057f96a5882125f2543d\": container with ID starting with 4c892fddc2576f6e5fd94895fcc21962381da0903f38057f96a5882125f2543d not found: ID does not exist" containerID="4c892fddc2576f6e5fd94895fcc21962381da0903f38057f96a5882125f2543d"
Nov 02 02:34:47.612157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.612087    4515 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c892fddc2576f6e5fd94895fcc21962381da0903f38057f96a5882125f2543d"} err="failed to get container status \"4c892fddc2576f6e5fd94895fcc21962381da0903f38057f96a5882125f2543d\": rpc error: code = NotFound desc = could not find container \"4c892fddc2576f6e5fd94895fcc21962381da0903f38057f96a5882125f2543d\": container with ID starting with 4c892fddc2576f6e5fd94895fcc21962381da0903f38057f96a5882125f2543d not found: ID does not exist"
Nov 02 02:34:47.768469 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.768383    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"]
Nov 02 02:34:47.770671 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:47.770644    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-66b669b4fd-j675q"]
Nov 02 02:34:48.951525 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:34:48.951492    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20040f1c-ca52-4c4c-af8f-fbe2ade458e4" path="/var/lib/kubelet/pods/20040f1c-ca52-4c4c-af8f-fbe2ade458e4/volumes"
Nov 02 02:35:05.826806 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.826772    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"]
Nov 02 02:35:05.827204 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.827073    4515 cpu_manager.go:414] "containerMap: removing container" podUID="20040f1c-ca52-4c4c-af8f-fbe2ade458e4" containerName="maas-api"
Nov 02 02:35:05.827204 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.827083    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20040f1c-ca52-4c4c-af8f-fbe2ade458e4" containerName="maas-api"
Nov 02 02:35:05.827204 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.827093    4515 cpu_manager.go:414] "containerMap: removing container" podUID="cd61c5ee-a5e5-4536-8103-a07153e0de16" containerName="main"
Nov 02 02:35:05.827204 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.827100    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd61c5ee-a5e5-4536-8103-a07153e0de16" containerName="main"
Nov 02 02:35:05.827204 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.827134    4515 cpu_manager.go:414] "containerMap: removing container" podUID="795be554-1e9e-4290-a45f-8bd4560e5ad8" containerName="main"
Nov 02 02:35:05.827204 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.827143    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="795be554-1e9e-4290-a45f-8bd4560e5ad8" containerName="main"
Nov 02 02:35:05.827204 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.827200    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="20040f1c-ca52-4c4c-af8f-fbe2ade458e4" containerName="maas-api"
Nov 02 02:35:05.827204 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.827206    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd61c5ee-a5e5-4536-8103-a07153e0de16" containerName="main"
Nov 02 02:35:05.827456 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.827215    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="795be554-1e9e-4290-a45f-8bd4560e5ad8" containerName="main"
Nov 02 02:35:05.829966 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.829949    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"
Nov 02 02:35:05.880664 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.880625    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/262bdc1d-e855-460c-9f42-fe2d8da3464d-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-71e54d09-vxww9\" (UID: \"262bdc1d-e855-460c-9f42-fe2d8da3464d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"
Nov 02 02:35:05.880829 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.880750    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/262bdc1d-e855-460c-9f42-fe2d8da3464d-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-71e54d09-vxww9\" (UID: \"262bdc1d-e855-460c-9f42-fe2d8da3464d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"
Nov 02 02:35:05.880829 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.880797    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg65h\" (UniqueName: \"kubernetes.io/projected/262bdc1d-e855-460c-9f42-fe2d8da3464d-kube-api-access-vg65h\") pod \"maas-api-revoke-keys-e2e-ait-71e54d09-vxww9\" (UID: \"262bdc1d-e855-460c-9f42-fe2d8da3464d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"
Nov 02 02:35:05.882625 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.882605    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 02:35:05.882720 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.882650    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w4rht\""
Nov 02 02:35:05.932236 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.932209    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 02 02:35:05.932236 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.932209    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 02 02:35:05.981244 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.981211    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/262bdc1d-e855-460c-9f42-fe2d8da3464d-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-71e54d09-vxww9\" (UID: \"262bdc1d-e855-460c-9f42-fe2d8da3464d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"
Nov 02 02:35:05.981347 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.981257    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg65h\" (UniqueName: \"kubernetes.io/projected/262bdc1d-e855-460c-9f42-fe2d8da3464d-kube-api-access-vg65h\") pod \"maas-api-revoke-keys-e2e-ait-71e54d09-vxww9\" (UID: \"262bdc1d-e855-460c-9f42-fe2d8da3464d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"
Nov 02 02:35:05.981347 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.981297    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/262bdc1d-e855-460c-9f42-fe2d8da3464d-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-71e54d09-vxww9\" (UID: \"262bdc1d-e855-460c-9f42-fe2d8da3464d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"
Nov 02 02:35:05.981857 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.981840    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/262bdc1d-e855-460c-9f42-fe2d8da3464d-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-71e54d09-vxww9\" (UID: \"262bdc1d-e855-460c-9f42-fe2d8da3464d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"
Nov 02 02:35:05.983013 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.982995    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"]
Nov 02 02:35:05.983569 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:05.983552    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/262bdc1d-e855-460c-9f42-fe2d8da3464d-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-71e54d09-vxww9\" (UID: \"262bdc1d-e855-460c-9f42-fe2d8da3464d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"
Nov 02 02:35:06.075513 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:06.075483    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg65h\" (UniqueName: \"kubernetes.io/projected/262bdc1d-e855-460c-9f42-fe2d8da3464d-kube-api-access-vg65h\") pod \"maas-api-revoke-keys-e2e-ait-71e54d09-vxww9\" (UID: \"262bdc1d-e855-460c-9f42-fe2d8da3464d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"
Nov 02 02:35:06.142525 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:06.142459    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"
Nov 02 02:35:06.519083 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:06.519050    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"]
Nov 02 02:35:06.521955 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:35:06.521928    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod262bdc1d_e855_460c_9f42_fe2d8da3464d.slice/crio-e3796b87ad8cb0982566fdeb73dfae81492c746464446402323bfe0896d9933a WatchSource:0}: Error finding container e3796b87ad8cb0982566fdeb73dfae81492c746464446402323bfe0896d9933a: Status 404 returned error can't find the container with id e3796b87ad8cb0982566fdeb73dfae81492c746464446402323bfe0896d9933a
Nov 02 02:35:06.680931 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:06.680901    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9" event={"ID":"262bdc1d-e855-460c-9f42-fe2d8da3464d","Type":"ContainerStarted","Data":"e3796b87ad8cb0982566fdeb73dfae81492c746464446402323bfe0896d9933a"}
Nov 02 02:35:07.686257 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:07.686223    4515 generic.go:356] "Generic (PLEG): container finished" podID="262bdc1d-e855-460c-9f42-fe2d8da3464d" containerID="92f80ff8e879fe837892734925c96965cbfd1a536137c63057bcbb022aabb80d" exitCode=0
Nov 02 02:35:07.686652 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:07.686268    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9" event={"ID":"262bdc1d-e855-460c-9f42-fe2d8da3464d","Type":"ContainerDied","Data":"92f80ff8e879fe837892734925c96965cbfd1a536137c63057bcbb022aabb80d"}
Nov 02 02:35:08.811110 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:08.811086    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"
Nov 02 02:35:08.904920 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:08.904881    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/262bdc1d-e855-460c-9f42-fe2d8da3464d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/262bdc1d-e855-460c-9f42-fe2d8da3464d-maas-api-openshift-ca\") pod \"262bdc1d-e855-460c-9f42-fe2d8da3464d\" (UID: \"262bdc1d-e855-460c-9f42-fe2d8da3464d\") "
Nov 02 02:35:08.905073 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:08.904962    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/262bdc1d-e855-460c-9f42-fe2d8da3464d-kube-api-access-vg65h\" (UniqueName: \"kubernetes.io/projected/262bdc1d-e855-460c-9f42-fe2d8da3464d-kube-api-access-vg65h\") pod \"262bdc1d-e855-460c-9f42-fe2d8da3464d\" (UID: \"262bdc1d-e855-460c-9f42-fe2d8da3464d\") "
Nov 02 02:35:08.905167 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:08.905071    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/262bdc1d-e855-460c-9f42-fe2d8da3464d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/262bdc1d-e855-460c-9f42-fe2d8da3464d-maas-api-xks-ca\") pod \"262bdc1d-e855-460c-9f42-fe2d8da3464d\" (UID: \"262bdc1d-e855-460c-9f42-fe2d8da3464d\") "
Nov 02 02:35:08.905363 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:08.905334    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/262bdc1d-e855-460c-9f42-fe2d8da3464d-maas-api-openshift-ca" pod "262bdc1d-e855-460c-9f42-fe2d8da3464d" (UID: "262bdc1d-e855-460c-9f42-fe2d8da3464d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:35:08.906970 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:08.906947    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/262bdc1d-e855-460c-9f42-fe2d8da3464d-maas-api-xks-ca" pod "262bdc1d-e855-460c-9f42-fe2d8da3464d" (UID: "262bdc1d-e855-460c-9f42-fe2d8da3464d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:35:08.907057 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:08.907032    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/262bdc1d-e855-460c-9f42-fe2d8da3464d-kube-api-access-vg65h" pod "262bdc1d-e855-460c-9f42-fe2d8da3464d" (UID: "262bdc1d-e855-460c-9f42-fe2d8da3464d"). InnerVolumeSpecName "kube-api-access-vg65h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:35:09.006108 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:09.006077    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vg65h\" (UniqueName: \"kubernetes.io/projected/262bdc1d-e855-460c-9f42-fe2d8da3464d-kube-api-access-vg65h\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:35:09.006108 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:09.006106    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/262bdc1d-e855-460c-9f42-fe2d8da3464d-maas-api-xks-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:35:09.006108 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:09.006133    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/262bdc1d-e855-460c-9f42-fe2d8da3464d-maas-api-openshift-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:35:09.694882 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:09.694840    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9" event={"ID":"262bdc1d-e855-460c-9f42-fe2d8da3464d","Type":"ContainerDied","Data":"e3796b87ad8cb0982566fdeb73dfae81492c746464446402323bfe0896d9933a"}
Nov 02 02:35:09.694882 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:09.694872    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"
Nov 02 02:35:09.695090 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:09.694898    4515 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3796b87ad8cb0982566fdeb73dfae81492c746464446402323bfe0896d9933a"
Nov 02 02:35:09.958699 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:35:09.958610    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:35:25.917499 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:25.917466    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"]
Nov 02 02:35:25.966842 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:25.966810    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71e54d09-vxww9"]
Nov 02 02:35:26.953610 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:26.953569    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="262bdc1d-e855-460c-9f42-fe2d8da3464d" path="/var/lib/kubelet/pods/262bdc1d-e855-460c-9f42-fe2d8da3464d/volumes"
Nov 02 02:35:28.595184 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.595150    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"]
Nov 02 02:35:28.595571 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.595431    4515 cpu_manager.go:414] "containerMap: removing container" podUID="262bdc1d-e855-460c-9f42-fe2d8da3464d" containerName="revoke-keys"
Nov 02 02:35:28.595571 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.595442    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="262bdc1d-e855-460c-9f42-fe2d8da3464d" containerName="revoke-keys"
Nov 02 02:35:28.595571 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.595528    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="262bdc1d-e855-460c-9f42-fe2d8da3464d" containerName="revoke-keys"
Nov 02 02:35:28.598221 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.598203    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.706074 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.706042    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-433aff-openshift-default-dockercfg-l2sb7\""
Nov 02 02:35:28.708280 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.708254    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"]
Nov 02 02:35:28.774647 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.774602    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-credential-socket\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.774837 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.774657    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33e970b6-208c-4000-b582-e0b8b2844504-istio-token\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.774837 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.774787    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-istio-data\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.774941 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.774854    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-workload-certs\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.774941 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.774886    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czdbb\" (UniqueName: \"kubernetes.io/projected/33e970b6-208c-4000-b582-e0b8b2844504-kube-api-access-czdbb\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.775038 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.774963    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33e970b6-208c-4000-b582-e0b8b2844504-istiod-ca-cert\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.775038 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.775003    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-istio-envoy\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.775139 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.775096    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33e970b6-208c-4000-b582-e0b8b2844504-istio-podinfo\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.775191 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.775155    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-workload-socket\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.875956 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.875869    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33e970b6-208c-4000-b582-e0b8b2844504-istio-podinfo\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.875956 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.875925    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-workload-socket\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.876218 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.875982    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-credential-socket\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.876218 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.876019    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33e970b6-208c-4000-b582-e0b8b2844504-istio-token\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.876218 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.876065    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-istio-data\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.876218 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.876099    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-workload-certs\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.876218 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.876135    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czdbb\" (UniqueName: \"kubernetes.io/projected/33e970b6-208c-4000-b582-e0b8b2844504-kube-api-access-czdbb\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.876218 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.876164    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33e970b6-208c-4000-b582-e0b8b2844504-istiod-ca-cert\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.876218 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.876187    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-istio-envoy\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.876592 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.876490    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-credential-socket\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.876592 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.876517    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-workload-socket\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.876592 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.876557    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-istio-data\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.876732 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.876590    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-workload-certs\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.877416 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.877388    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33e970b6-208c-4000-b582-e0b8b2844504-istiod-ca-cert\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.878512 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.878491    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33e970b6-208c-4000-b582-e0b8b2844504-istio-podinfo\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.878605 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.878550    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-istio-envoy\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.938336 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.938309    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czdbb\" (UniqueName: \"kubernetes.io/projected/33e970b6-208c-4000-b582-e0b8b2844504-kube-api-access-czdbb\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:28.948199 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:28.948173    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33e970b6-208c-4000-b582-e0b8b2844504-istio-token\") pod \"e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") " pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:29.211474 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:29.211442    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:29.642576 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:29.642486    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"]
Nov 02 02:35:29.645547 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:35:29.645512    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33e970b6_208c_4000_b582_e0b8b2844504.slice/crio-24bbfbcc48ac9dca6be44f1093ede234ed4cd91c11246904349792985ef9245b WatchSource:0}: Error finding container 24bbfbcc48ac9dca6be44f1093ede234ed4cd91c11246904349792985ef9245b: Status 404 returned error can't find the container with id 24bbfbcc48ac9dca6be44f1093ede234ed4cd91c11246904349792985ef9245b
Nov 02 02:35:29.647664 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:29.647629    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:35:29.647741 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:29.647699    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:35:29.647741 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:29.647728    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:35:29.776385 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:29.776347    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr" event={"ID":"33e970b6-208c-4000-b582-e0b8b2844504","Type":"ContainerStarted","Data":"60dc10254a4d84c22ccc92205e0f88ace84ae6859e424c8ac5a0dda17a42e2d0"}
Nov 02 02:35:29.776581 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:29.776400    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr" event={"ID":"33e970b6-208c-4000-b582-e0b8b2844504","Type":"ContainerStarted","Data":"24bbfbcc48ac9dca6be44f1093ede234ed4cd91c11246904349792985ef9245b"}
Nov 02 02:35:29.985313 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:29.985262    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr" podStartSLOduration=1.985248495 podStartE2EDuration="1.985248495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:35:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:35:29.935468978 +0000 UTC m=+3819.147838801" watchObservedRunningTime="2026-11-02 02:35:29.985248495 +0000 UTC m=+3819.197618302"
Nov 02 02:35:30.211662 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:30.211621    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:30.216738 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:30.216708    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:30.784820 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:30.784795    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:30.785951 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:30.785928    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:39.958247 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:35:39.958211    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:35:51.886093 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:51.886060    4515 generic.go:356] "Generic (PLEG): container finished" podID="33e970b6-208c-4000-b582-e0b8b2844504" containerID="60dc10254a4d84c22ccc92205e0f88ace84ae6859e424c8ac5a0dda17a42e2d0" exitCode=0
Nov 02 02:35:51.886609 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:51.886103    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr" event={"ID":"33e970b6-208c-4000-b582-e0b8b2844504","Type":"ContainerDied","Data":"60dc10254a4d84c22ccc92205e0f88ace84ae6859e424c8ac5a0dda17a42e2d0"}
Nov 02 02:35:51.886609 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:51.886564    4515 scope.go:122] "RemoveContainer" containerID="60dc10254a4d84c22ccc92205e0f88ace84ae6859e424c8ac5a0dda17a42e2d0"
Nov 02 02:35:51.887734 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:51.887703    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:35:51.887836 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:51.887791    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:35:51.887914 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:51.887835    4515 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 02:35:52.892433 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:52.892394    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr" event={"ID":"33e970b6-208c-4000-b582-e0b8b2844504","Type":"ContainerStarted","Data":"8a87f0e494b82be2a103afb99f4aad91c31ce49b959e0fd9b31818a6a4b60985"}
Nov 02 02:35:53.211660 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:53.211620    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:53.211903 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:53.211879    4515 patch_prober.go:29] interesting pod/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 02:35:53.211952 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:53.211937    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr" podUID="33e970b6-208c-4000-b582-e0b8b2844504" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 02:35:54.212651 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:54.212609    4515 patch_prober.go:29] interesting pod/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 02:35:54.213193 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:54.212691    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr" podUID="33e970b6-208c-4000-b582-e0b8b2844504" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 02:35:55.212733 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:55.212695    4515 patch_prober.go:29] interesting pod/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 02:35:55.213165 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:55.212763    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr" podUID="33e970b6-208c-4000-b582-e0b8b2844504" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 02:35:56.212703 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:56.212662    4515 patch_prober.go:29] interesting pod/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 02:35:56.212946 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:56.212733    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr" podUID="33e970b6-208c-4000-b582-e0b8b2844504" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 02:35:57.212775 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:57.212735    4515 patch_prober.go:29] interesting pod/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 02:35:57.213056 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:57.212787    4515 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr" podUID="33e970b6-208c-4000-b582-e0b8b2844504" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 02:35:58.216543 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:58.216512    4515 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:58.217016 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:58.216901    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:35:58.217649 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:35:58.217627    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:36:09.959254 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:36:09.959214    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:36:39.958111 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:36:39.958075    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:36:51.330603 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:36:51.330568    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 02:36:51.335220 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:36:51.335202    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gs99s_62f7a5eb-5a3c-4bd5-a0cd-486539e66c82/ovn-acl-logging/0.log"
Nov 02 02:37:09.957939 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:37:09.957856    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:37:13.188090 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.188056    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"]
Nov 02 02:37:13.191226 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.191208    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"
Nov 02 02:37:13.244928 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.244896    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 02 02:37:13.294262 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.294223    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 02:37:13.294262 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.294254    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 02 02:37:13.294519 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.294300    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w4rht\""
Nov 02 02:37:13.295691 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.295671    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"]
Nov 02 02:37:13.326499 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.326467    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2200e453-c3e6-4eda-a498-3428805f1dcc-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5\" (UID: \"2200e453-c3e6-4eda-a498-3428805f1dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"
Nov 02 02:37:13.326637 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.326584    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2200e453-c3e6-4eda-a498-3428805f1dcc-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5\" (UID: \"2200e453-c3e6-4eda-a498-3428805f1dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"
Nov 02 02:37:13.326637 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.326621    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d696m\" (UniqueName: \"kubernetes.io/projected/2200e453-c3e6-4eda-a498-3428805f1dcc-kube-api-access-d696m\") pod \"maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5\" (UID: \"2200e453-c3e6-4eda-a498-3428805f1dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"
Nov 02 02:37:13.427810 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.427769    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2200e453-c3e6-4eda-a498-3428805f1dcc-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5\" (UID: \"2200e453-c3e6-4eda-a498-3428805f1dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"
Nov 02 02:37:13.428021 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.427850    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2200e453-c3e6-4eda-a498-3428805f1dcc-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5\" (UID: \"2200e453-c3e6-4eda-a498-3428805f1dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"
Nov 02 02:37:13.428021 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.427871    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d696m\" (UniqueName: \"kubernetes.io/projected/2200e453-c3e6-4eda-a498-3428805f1dcc-kube-api-access-d696m\") pod \"maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5\" (UID: \"2200e453-c3e6-4eda-a498-3428805f1dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"
Nov 02 02:37:13.428539 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.428519    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2200e453-c3e6-4eda-a498-3428805f1dcc-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5\" (UID: \"2200e453-c3e6-4eda-a498-3428805f1dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"
Nov 02 02:37:13.430185 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.430165    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2200e453-c3e6-4eda-a498-3428805f1dcc-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5\" (UID: \"2200e453-c3e6-4eda-a498-3428805f1dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"
Nov 02 02:37:13.489566 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.489491    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d696m\" (UniqueName: \"kubernetes.io/projected/2200e453-c3e6-4eda-a498-3428805f1dcc-kube-api-access-d696m\") pod \"maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5\" (UID: \"2200e453-c3e6-4eda-a498-3428805f1dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"
Nov 02 02:37:13.503264 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.503243    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"
Nov 02 02:37:13.875603 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:13.875524    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"]
Nov 02 02:37:13.878743 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:37:13.878718    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2200e453_c3e6_4eda_a498_3428805f1dcc.slice/crio-abe97bb8a581e04f783184e46074b9a23d3358633803422103873e1ade628037 WatchSource:0}: Error finding container abe97bb8a581e04f783184e46074b9a23d3358633803422103873e1ade628037: Status 404 returned error can't find the container with id abe97bb8a581e04f783184e46074b9a23d3358633803422103873e1ade628037
Nov 02 02:37:14.205858 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:14.205813    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5" event={"ID":"2200e453-c3e6-4eda-a498-3428805f1dcc","Type":"ContainerStarted","Data":"1d39356659991dcd1b9a7dd4044c29fff51ddc1a6e8de3e907d099a367c5c5bb"}
Nov 02 02:37:14.206408 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:14.205918    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5" event={"ID":"2200e453-c3e6-4eda-a498-3428805f1dcc","Type":"ContainerStarted","Data":"abe97bb8a581e04f783184e46074b9a23d3358633803422103873e1ade628037"}
Nov 02 02:37:14.322877 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:14.322822    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5" podStartSLOduration=1.322806745 podStartE2EDuration="1.322806745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:37:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:37:14.321533385 +0000 UTC m=+3923.533903194" watchObservedRunningTime="2026-11-02 02:37:14.322806745 +0000 UTC m=+3923.535176552"
Nov 02 02:37:15.210955 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:15.210920    4515 generic.go:356] "Generic (PLEG): container finished" podID="2200e453-c3e6-4eda-a498-3428805f1dcc" containerID="1d39356659991dcd1b9a7dd4044c29fff51ddc1a6e8de3e907d099a367c5c5bb" exitCode=0
Nov 02 02:37:15.211419 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:15.211014    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5" event={"ID":"2200e453-c3e6-4eda-a498-3428805f1dcc","Type":"ContainerDied","Data":"1d39356659991dcd1b9a7dd4044c29fff51ddc1a6e8de3e907d099a367c5c5bb"}
Nov 02 02:37:16.342570 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:16.342542    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"
Nov 02 02:37:16.350831 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:16.350800    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2200e453-c3e6-4eda-a498-3428805f1dcc-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2200e453-c3e6-4eda-a498-3428805f1dcc-maas-api-xks-ca\") pod \"2200e453-c3e6-4eda-a498-3428805f1dcc\" (UID: \"2200e453-c3e6-4eda-a498-3428805f1dcc\") "
Nov 02 02:37:16.350982 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:16.350861    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2200e453-c3e6-4eda-a498-3428805f1dcc-kube-api-access-d696m\" (UniqueName: \"kubernetes.io/projected/2200e453-c3e6-4eda-a498-3428805f1dcc-kube-api-access-d696m\") pod \"2200e453-c3e6-4eda-a498-3428805f1dcc\" (UID: \"2200e453-c3e6-4eda-a498-3428805f1dcc\") "
Nov 02 02:37:16.350982 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:16.350898    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2200e453-c3e6-4eda-a498-3428805f1dcc-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2200e453-c3e6-4eda-a498-3428805f1dcc-maas-api-openshift-ca\") pod \"2200e453-c3e6-4eda-a498-3428805f1dcc\" (UID: \"2200e453-c3e6-4eda-a498-3428805f1dcc\") "
Nov 02 02:37:16.351320 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:16.351296    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2200e453-c3e6-4eda-a498-3428805f1dcc-maas-api-openshift-ca" pod "2200e453-c3e6-4eda-a498-3428805f1dcc" (UID: "2200e453-c3e6-4eda-a498-3428805f1dcc"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:37:16.352946 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:16.352922    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2200e453-c3e6-4eda-a498-3428805f1dcc-kube-api-access-d696m" pod "2200e453-c3e6-4eda-a498-3428805f1dcc" (UID: "2200e453-c3e6-4eda-a498-3428805f1dcc"). InnerVolumeSpecName "kube-api-access-d696m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:37:16.353010 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:16.352942    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2200e453-c3e6-4eda-a498-3428805f1dcc-maas-api-xks-ca" pod "2200e453-c3e6-4eda-a498-3428805f1dcc" (UID: "2200e453-c3e6-4eda-a498-3428805f1dcc"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:37:16.451731 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:16.451676    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d696m\" (UniqueName: \"kubernetes.io/projected/2200e453-c3e6-4eda-a498-3428805f1dcc-kube-api-access-d696m\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:37:16.451731 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:16.451725    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2200e453-c3e6-4eda-a498-3428805f1dcc-maas-api-openshift-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:37:16.451731 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:16.451739    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2200e453-c3e6-4eda-a498-3428805f1dcc-maas-api-xks-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:37:17.222146 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:17.222037    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5" event={"ID":"2200e453-c3e6-4eda-a498-3428805f1dcc","Type":"ContainerDied","Data":"abe97bb8a581e04f783184e46074b9a23d3358633803422103873e1ade628037"}
Nov 02 02:37:17.222146 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:17.222093    4515 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="abe97bb8a581e04f783184e46074b9a23d3358633803422103873e1ade628037"
Nov 02 02:37:17.222146 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:17.222051    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"
Nov 02 02:37:37.752362 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:37.752320    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"]
Nov 02 02:37:37.800740 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:37.800702    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-433aff-j9vz5"]
Nov 02 02:37:38.952425 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:38.952389    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2200e453-c3e6-4eda-a498-3428805f1dcc" path="/var/lib/kubelet/pods/2200e453-c3e6-4eda-a498-3428805f1dcc/volumes"
Nov 02 02:37:39.957924 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:37:39.957888    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:37:45.199365 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:45.199323    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"]
Nov 02 02:37:45.199894 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:45.199538    4515 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr" podUID="33e970b6-208c-4000-b582-e0b8b2844504" containerName="istio-proxy" containerID="cri-o://8a87f0e494b82be2a103afb99f4aad91c31ce49b959e0fd9b31818a6a4b60985" gracePeriod=30
Nov 02 02:37:50.362544 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.362504    4515 generic.go:356] "Generic (PLEG): container finished" podID="33e970b6-208c-4000-b582-e0b8b2844504" containerID="8a87f0e494b82be2a103afb99f4aad91c31ce49b959e0fd9b31818a6a4b60985" exitCode=0
Nov 02 02:37:50.362948 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.362575    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr" event={"ID":"33e970b6-208c-4000-b582-e0b8b2844504","Type":"ContainerDied","Data":"8a87f0e494b82be2a103afb99f4aad91c31ce49b959e0fd9b31818a6a4b60985"}
Nov 02 02:37:50.362948 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.362634    4515 scope.go:122] "RemoveContainer" containerID="60dc10254a4d84c22ccc92205e0f88ace84ae6859e424c8ac5a0dda17a42e2d0"
Nov 02 02:37:50.462677 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.462655    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:37:50.540219 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.540158    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33e970b6-208c-4000-b582-e0b8b2844504-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33e970b6-208c-4000-b582-e0b8b2844504-istiod-ca-cert\") pod \"33e970b6-208c-4000-b582-e0b8b2844504\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") "
Nov 02 02:37:50.540219 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.540198    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-workload-socket\") pod \"33e970b6-208c-4000-b582-e0b8b2844504\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") "
Nov 02 02:37:50.540378 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.540227    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33e970b6-208c-4000-b582-e0b8b2844504-istio-token\" (UniqueName: \"kubernetes.io/projected/33e970b6-208c-4000-b582-e0b8b2844504-istio-token\") pod \"33e970b6-208c-4000-b582-e0b8b2844504\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") "
Nov 02 02:37:50.540481 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.540395    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/33e970b6-208c-4000-b582-e0b8b2844504-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33e970b6-208c-4000-b582-e0b8b2844504-istio-podinfo\") pod \"33e970b6-208c-4000-b582-e0b8b2844504\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") "
Nov 02 02:37:50.540481 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.540449    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-workload-certs\") pod \"33e970b6-208c-4000-b582-e0b8b2844504\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") "
Nov 02 02:37:50.540598 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.540490    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-workload-socket" pod "33e970b6-208c-4000-b582-e0b8b2844504" (UID: "33e970b6-208c-4000-b582-e0b8b2844504"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:37:50.540598 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.540526    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-credential-socket\") pod \"33e970b6-208c-4000-b582-e0b8b2844504\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") "
Nov 02 02:37:50.540598 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.540543    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33e970b6-208c-4000-b582-e0b8b2844504-istiod-ca-cert" pod "33e970b6-208c-4000-b582-e0b8b2844504" (UID: "33e970b6-208c-4000-b582-e0b8b2844504"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:37:50.540598 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.540556    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-istio-envoy\") pod \"33e970b6-208c-4000-b582-e0b8b2844504\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") "
Nov 02 02:37:50.540598 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.540596    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-istio-data\") pod \"33e970b6-208c-4000-b582-e0b8b2844504\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") "
Nov 02 02:37:50.540880 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.540635    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33e970b6-208c-4000-b582-e0b8b2844504-kube-api-access-czdbb\" (UniqueName: \"kubernetes.io/projected/33e970b6-208c-4000-b582-e0b8b2844504-kube-api-access-czdbb\") pod \"33e970b6-208c-4000-b582-e0b8b2844504\" (UID: \"33e970b6-208c-4000-b582-e0b8b2844504\") "
Nov 02 02:37:50.540880 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.540775    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-workload-certs" pod "33e970b6-208c-4000-b582-e0b8b2844504" (UID: "33e970b6-208c-4000-b582-e0b8b2844504"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:37:50.540880 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.540824    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-credential-socket" pod "33e970b6-208c-4000-b582-e0b8b2844504" (UID: "33e970b6-208c-4000-b582-e0b8b2844504"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:37:50.540880 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.540866    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-istio-data" pod "33e970b6-208c-4000-b582-e0b8b2844504" (UID: "33e970b6-208c-4000-b582-e0b8b2844504"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:37:50.541093 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.540976    4515 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-credential-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:37:50.541093 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.540991    4515 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-istio-data\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:37:50.541093 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.541000    4515 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33e970b6-208c-4000-b582-e0b8b2844504-istiod-ca-cert\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:37:50.541093 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.541010    4515 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-workload-socket\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:37:50.541093 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.541018    4515 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-workload-certs\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:37:50.542439 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.542411    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33e970b6-208c-4000-b582-e0b8b2844504-istio-token" pod "33e970b6-208c-4000-b582-e0b8b2844504" (UID: "33e970b6-208c-4000-b582-e0b8b2844504"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:37:50.542580 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.542542    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33e970b6-208c-4000-b582-e0b8b2844504-istio-podinfo" pod "33e970b6-208c-4000-b582-e0b8b2844504" (UID: "33e970b6-208c-4000-b582-e0b8b2844504"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 02:37:50.542711 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.542694    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33e970b6-208c-4000-b582-e0b8b2844504-kube-api-access-czdbb" pod "33e970b6-208c-4000-b582-e0b8b2844504" (UID: "33e970b6-208c-4000-b582-e0b8b2844504"). InnerVolumeSpecName "kube-api-access-czdbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:37:50.542786 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.542771    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-istio-envoy" pod "33e970b6-208c-4000-b582-e0b8b2844504" (UID: "33e970b6-208c-4000-b582-e0b8b2844504"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:37:50.641497 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.641466    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czdbb\" (UniqueName: \"kubernetes.io/projected/33e970b6-208c-4000-b582-e0b8b2844504-kube-api-access-czdbb\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:37:50.641497 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.641492    4515 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33e970b6-208c-4000-b582-e0b8b2844504-istio-token\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:37:50.641497 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.641503    4515 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33e970b6-208c-4000-b582-e0b8b2844504-istio-podinfo\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:37:50.641718 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:50.641511    4515 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33e970b6-208c-4000-b582-e0b8b2844504-istio-envoy\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:37:51.367942 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:51.367864    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"
Nov 02 02:37:51.367942 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:51.367878    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr" event={"ID":"33e970b6-208c-4000-b582-e0b8b2844504","Type":"ContainerDied","Data":"24bbfbcc48ac9dca6be44f1093ede234ed4cd91c11246904349792985ef9245b"}
Nov 02 02:37:51.368433 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:51.367962    4515 scope.go:122] "RemoveContainer" containerID="8a87f0e494b82be2a103afb99f4aad91c31ce49b959e0fd9b31818a6a4b60985"
Nov 02 02:37:51.541954 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:51.541921    4515 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"]
Nov 02 02:37:51.591570 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:51.591540    4515 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-433aff-openshift-default-65746565cc-wv4nr"]
Nov 02 02:37:52.951661 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:37:52.951620    4515 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33e970b6-208c-4000-b582-e0b8b2844504" path="/var/lib/kubelet/pods/33e970b6-208c-4000-b582-e0b8b2844504/volumes"
Nov 02 02:38:10.012798 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:38:10.012763    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:38:15.813186 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.813151    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9"]
Nov 02 02:38:15.813611 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.813443    4515 cpu_manager.go:414] "containerMap: removing container" podUID="2200e453-c3e6-4eda-a498-3428805f1dcc" containerName="revoke-keys"
Nov 02 02:38:15.813611 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.813455    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2200e453-c3e6-4eda-a498-3428805f1dcc" containerName="revoke-keys"
Nov 02 02:38:15.813611 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.813468    4515 cpu_manager.go:414] "containerMap: removing container" podUID="33e970b6-208c-4000-b582-e0b8b2844504" containerName="istio-proxy"
Nov 02 02:38:15.813611 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.813475    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33e970b6-208c-4000-b582-e0b8b2844504" containerName="istio-proxy"
Nov 02 02:38:15.813611 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.813533    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="2200e453-c3e6-4eda-a498-3428805f1dcc" containerName="revoke-keys"
Nov 02 02:38:15.813611 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.813541    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="33e970b6-208c-4000-b582-e0b8b2844504" containerName="istio-proxy"
Nov 02 02:38:15.813611 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.813549    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="33e970b6-208c-4000-b582-e0b8b2844504" containerName="istio-proxy"
Nov 02 02:38:15.816410 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.816395    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9"
Nov 02 02:38:15.868889 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.868860    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 02:38:15.868889 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.868876    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 02 02:38:15.883499 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.883466    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c2ae597-0a15-450a-8fc6-0b71488ba58a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-c4kf9\" (UID: \"8c2ae597-0a15-450a-8fc6-0b71488ba58a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9"
Nov 02 02:38:15.883653 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.883516    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88rq6\" (UniqueName: \"kubernetes.io/projected/8c2ae597-0a15-450a-8fc6-0b71488ba58a-kube-api-access-88rq6\") pod \"maas-api-revoke-sub-f03629db-c4kf9\" (UID: \"8c2ae597-0a15-450a-8fc6-0b71488ba58a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9"
Nov 02 02:38:15.883696 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.883654    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c2ae597-0a15-450a-8fc6-0b71488ba58a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-c4kf9\" (UID: \"8c2ae597-0a15-450a-8fc6-0b71488ba58a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9"
Nov 02 02:38:15.918640 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.918615    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 02 02:38:15.918726 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.918646    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w4rht\""
Nov 02 02:38:15.919861 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.919843    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9"]
Nov 02 02:38:15.984440 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.984407    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c2ae597-0a15-450a-8fc6-0b71488ba58a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-c4kf9\" (UID: \"8c2ae597-0a15-450a-8fc6-0b71488ba58a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9"
Nov 02 02:38:15.984623 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.984467    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c2ae597-0a15-450a-8fc6-0b71488ba58a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-c4kf9\" (UID: \"8c2ae597-0a15-450a-8fc6-0b71488ba58a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9"
Nov 02 02:38:15.984623 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.984492    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88rq6\" (UniqueName: \"kubernetes.io/projected/8c2ae597-0a15-450a-8fc6-0b71488ba58a-kube-api-access-88rq6\") pod \"maas-api-revoke-sub-f03629db-c4kf9\" (UID: \"8c2ae597-0a15-450a-8fc6-0b71488ba58a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9"
Nov 02 02:38:15.985194 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.985163    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c2ae597-0a15-450a-8fc6-0b71488ba58a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-c4kf9\" (UID: \"8c2ae597-0a15-450a-8fc6-0b71488ba58a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9"
Nov 02 02:38:15.986877 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:15.986848    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c2ae597-0a15-450a-8fc6-0b71488ba58a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-c4kf9\" (UID: \"8c2ae597-0a15-450a-8fc6-0b71488ba58a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9"
Nov 02 02:38:16.046060 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:16.046021    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88rq6\" (UniqueName: \"kubernetes.io/projected/8c2ae597-0a15-450a-8fc6-0b71488ba58a-kube-api-access-88rq6\") pod \"maas-api-revoke-sub-f03629db-c4kf9\" (UID: \"8c2ae597-0a15-450a-8fc6-0b71488ba58a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9"
Nov 02 02:38:16.128652 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:16.128580    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9"
Nov 02 02:38:16.498996 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:16.498965    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9"]
Nov 02 02:38:16.502078 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:38:16.502045    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c2ae597_0a15_450a_8fc6_0b71488ba58a.slice/crio-d6b0168bbf752cfa4234f2cbec727ead2cbe984d26e7074076490d115b4b1181 WatchSource:0}: Error finding container d6b0168bbf752cfa4234f2cbec727ead2cbe984d26e7074076490d115b4b1181: Status 404 returned error can't find the container with id d6b0168bbf752cfa4234f2cbec727ead2cbe984d26e7074076490d115b4b1181
Nov 02 02:38:17.493645 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:17.493611    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9" event={"ID":"8c2ae597-0a15-450a-8fc6-0b71488ba58a","Type":"ContainerStarted","Data":"56d8c0e8394f6bcdede6861fedf2468727685a45528c679dc826050250e06cc0"}
Nov 02 02:38:17.494020 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:17.493672    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9" event={"ID":"8c2ae597-0a15-450a-8fc6-0b71488ba58a","Type":"ContainerStarted","Data":"d6b0168bbf752cfa4234f2cbec727ead2cbe984d26e7074076490d115b4b1181"}
Nov 02 02:38:17.609065 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:17.609000    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9" podStartSLOduration=2.608981878 podStartE2EDuration="2.608981878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:38:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:38:17.608018987 +0000 UTC m=+3986.820388799" watchObservedRunningTime="2026-11-02 02:38:17.608981878 +0000 UTC m=+3986.821351688"
Nov 02 02:38:18.498595 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:18.498559    4515 generic.go:356] "Generic (PLEG): container finished" podID="8c2ae597-0a15-450a-8fc6-0b71488ba58a" containerID="56d8c0e8394f6bcdede6861fedf2468727685a45528c679dc826050250e06cc0" exitCode=0
Nov 02 02:38:18.499010 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:18.498639    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9" event={"ID":"8c2ae597-0a15-450a-8fc6-0b71488ba58a","Type":"ContainerDied","Data":"56d8c0e8394f6bcdede6861fedf2468727685a45528c679dc826050250e06cc0"}
Nov 02 02:38:19.628077 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:19.628052    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9"
Nov 02 02:38:19.719453 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:19.719422    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c2ae597-0a15-450a-8fc6-0b71488ba58a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c2ae597-0a15-450a-8fc6-0b71488ba58a-maas-api-xks-ca\") pod \"8c2ae597-0a15-450a-8fc6-0b71488ba58a\" (UID: \"8c2ae597-0a15-450a-8fc6-0b71488ba58a\") "
Nov 02 02:38:19.719644 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:19.719482    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c2ae597-0a15-450a-8fc6-0b71488ba58a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c2ae597-0a15-450a-8fc6-0b71488ba58a-maas-api-openshift-ca\") pod \"8c2ae597-0a15-450a-8fc6-0b71488ba58a\" (UID: \"8c2ae597-0a15-450a-8fc6-0b71488ba58a\") "
Nov 02 02:38:19.719644 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:19.719530    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c2ae597-0a15-450a-8fc6-0b71488ba58a-kube-api-access-88rq6\" (UniqueName: \"kubernetes.io/projected/8c2ae597-0a15-450a-8fc6-0b71488ba58a-kube-api-access-88rq6\") pod \"8c2ae597-0a15-450a-8fc6-0b71488ba58a\" (UID: \"8c2ae597-0a15-450a-8fc6-0b71488ba58a\") "
Nov 02 02:38:19.719916 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:19.719885    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c2ae597-0a15-450a-8fc6-0b71488ba58a-maas-api-openshift-ca" pod "8c2ae597-0a15-450a-8fc6-0b71488ba58a" (UID: "8c2ae597-0a15-450a-8fc6-0b71488ba58a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:38:19.721573 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:19.721552    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c2ae597-0a15-450a-8fc6-0b71488ba58a-maas-api-xks-ca" pod "8c2ae597-0a15-450a-8fc6-0b71488ba58a" (UID: "8c2ae597-0a15-450a-8fc6-0b71488ba58a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:38:19.721665 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:19.721645    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c2ae597-0a15-450a-8fc6-0b71488ba58a-kube-api-access-88rq6" pod "8c2ae597-0a15-450a-8fc6-0b71488ba58a" (UID: "8c2ae597-0a15-450a-8fc6-0b71488ba58a"). InnerVolumeSpecName "kube-api-access-88rq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:38:19.820562 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:19.820481    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c2ae597-0a15-450a-8fc6-0b71488ba58a-maas-api-xks-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:38:19.820562 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:19.820536    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c2ae597-0a15-450a-8fc6-0b71488ba58a-maas-api-openshift-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:38:19.820562 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:19.820551    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88rq6\" (UniqueName: \"kubernetes.io/projected/8c2ae597-0a15-450a-8fc6-0b71488ba58a-kube-api-access-88rq6\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:38:20.507527 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:20.507496    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9"
Nov 02 02:38:20.507527 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:20.507511    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-c4kf9" event={"ID":"8c2ae597-0a15-450a-8fc6-0b71488ba58a","Type":"ContainerDied","Data":"d6b0168bbf752cfa4234f2cbec727ead2cbe984d26e7074076490d115b4b1181"}
Nov 02 02:38:20.507832 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:20.507557    4515 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6b0168bbf752cfa4234f2cbec727ead2cbe984d26e7074076490d115b4b1181"
Nov 02 02:38:36.526998 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.526908    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv"]
Nov 02 02:38:36.527485 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.527362    4515 cpu_manager.go:414] "containerMap: removing container" podUID="33e970b6-208c-4000-b582-e0b8b2844504" containerName="istio-proxy"
Nov 02 02:38:36.527485 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.527380    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33e970b6-208c-4000-b582-e0b8b2844504" containerName="istio-proxy"
Nov 02 02:38:36.527485 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.527416    4515 cpu_manager.go:414] "containerMap: removing container" podUID="8c2ae597-0a15-450a-8fc6-0b71488ba58a" containerName="revoke-keys"
Nov 02 02:38:36.527485 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.527424    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c2ae597-0a15-450a-8fc6-0b71488ba58a" containerName="revoke-keys"
Nov 02 02:38:36.527485 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.527485    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c2ae597-0a15-450a-8fc6-0b71488ba58a" containerName="revoke-keys"
Nov 02 02:38:36.530228 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.530197    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv"
Nov 02 02:38:36.582453 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.582422    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 02 02:38:36.582453 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.582430    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 02:38:36.631996 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.631967    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 02 02:38:36.632285 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.632267    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w4rht\""
Nov 02 02:38:36.633551 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.633530    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv"]
Nov 02 02:38:36.699428 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.699398    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e49372a7-bb92-4daf-945a-ee03283fe26d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-2q8qv\" (UID: \"e49372a7-bb92-4daf-945a-ee03283fe26d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv"
Nov 02 02:38:36.699570 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.699470    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdrkz\" (UniqueName: \"kubernetes.io/projected/e49372a7-bb92-4daf-945a-ee03283fe26d-kube-api-access-pdrkz\") pod \"maas-api-revoke-sub-0165d7d2-2q8qv\" (UID: \"e49372a7-bb92-4daf-945a-ee03283fe26d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv"
Nov 02 02:38:36.699623 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.699575    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e49372a7-bb92-4daf-945a-ee03283fe26d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-2q8qv\" (UID: \"e49372a7-bb92-4daf-945a-ee03283fe26d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv"
Nov 02 02:38:36.800477 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.800406    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e49372a7-bb92-4daf-945a-ee03283fe26d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-2q8qv\" (UID: \"e49372a7-bb92-4daf-945a-ee03283fe26d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv"
Nov 02 02:38:36.800477 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.800462    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e49372a7-bb92-4daf-945a-ee03283fe26d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-2q8qv\" (UID: \"e49372a7-bb92-4daf-945a-ee03283fe26d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv"
Nov 02 02:38:36.800657 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.800496    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdrkz\" (UniqueName: \"kubernetes.io/projected/e49372a7-bb92-4daf-945a-ee03283fe26d-kube-api-access-pdrkz\") pod \"maas-api-revoke-sub-0165d7d2-2q8qv\" (UID: \"e49372a7-bb92-4daf-945a-ee03283fe26d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv"
Nov 02 02:38:36.801201 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.801178    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e49372a7-bb92-4daf-945a-ee03283fe26d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-2q8qv\" (UID: \"e49372a7-bb92-4daf-945a-ee03283fe26d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv"
Nov 02 02:38:36.802791 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.802768    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e49372a7-bb92-4daf-945a-ee03283fe26d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-2q8qv\" (UID: \"e49372a7-bb92-4daf-945a-ee03283fe26d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv"
Nov 02 02:38:36.861363 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:36.861336    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdrkz\" (UniqueName: \"kubernetes.io/projected/e49372a7-bb92-4daf-945a-ee03283fe26d-kube-api-access-pdrkz\") pod \"maas-api-revoke-sub-0165d7d2-2q8qv\" (UID: \"e49372a7-bb92-4daf-945a-ee03283fe26d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv"
Nov 02 02:38:37.145876 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:37.145805    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv"
Nov 02 02:38:37.514317 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:37.514284    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv"]
Nov 02 02:38:37.517706 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:38:37.517680    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode49372a7_bb92_4daf_945a_ee03283fe26d.slice/crio-70f721334c425fd91c997d812a40f622d0d8b5e027fec7dc6b0ccd501f052328 WatchSource:0}: Error finding container 70f721334c425fd91c997d812a40f622d0d8b5e027fec7dc6b0ccd501f052328: Status 404 returned error can't find the container with id 70f721334c425fd91c997d812a40f622d0d8b5e027fec7dc6b0ccd501f052328
Nov 02 02:38:37.588157 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:37.588125    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv" event={"ID":"e49372a7-bb92-4daf-945a-ee03283fe26d","Type":"ContainerStarted","Data":"70f721334c425fd91c997d812a40f622d0d8b5e027fec7dc6b0ccd501f052328"}
Nov 02 02:38:38.594428 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:38.594345    4515 generic.go:356] "Generic (PLEG): container finished" podID="e49372a7-bb92-4daf-945a-ee03283fe26d" containerID="88dda881a1335bcb9b99676acb80a7e5d259928340ba6ca6b2a5d4019aaf85cf" exitCode=0
Nov 02 02:38:38.594793 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:38.594434    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv" event={"ID":"e49372a7-bb92-4daf-945a-ee03283fe26d","Type":"ContainerDied","Data":"88dda881a1335bcb9b99676acb80a7e5d259928340ba6ca6b2a5d4019aaf85cf"}
Nov 02 02:38:39.724130 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:39.724089    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv"
Nov 02 02:38:39.824781 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:39.824746    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e49372a7-bb92-4daf-945a-ee03283fe26d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e49372a7-bb92-4daf-945a-ee03283fe26d-maas-api-openshift-ca\") pod \"e49372a7-bb92-4daf-945a-ee03283fe26d\" (UID: \"e49372a7-bb92-4daf-945a-ee03283fe26d\") "
Nov 02 02:38:39.824955 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:39.824790    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e49372a7-bb92-4daf-945a-ee03283fe26d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e49372a7-bb92-4daf-945a-ee03283fe26d-maas-api-xks-ca\") pod \"e49372a7-bb92-4daf-945a-ee03283fe26d\" (UID: \"e49372a7-bb92-4daf-945a-ee03283fe26d\") "
Nov 02 02:38:39.824955 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:39.824866    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e49372a7-bb92-4daf-945a-ee03283fe26d-kube-api-access-pdrkz\" (UniqueName: \"kubernetes.io/projected/e49372a7-bb92-4daf-945a-ee03283fe26d-kube-api-access-pdrkz\") pod \"e49372a7-bb92-4daf-945a-ee03283fe26d\" (UID: \"e49372a7-bb92-4daf-945a-ee03283fe26d\") "
Nov 02 02:38:39.825166 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:39.825146    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e49372a7-bb92-4daf-945a-ee03283fe26d-maas-api-openshift-ca" pod "e49372a7-bb92-4daf-945a-ee03283fe26d" (UID: "e49372a7-bb92-4daf-945a-ee03283fe26d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:38:39.826908 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:39.826873    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e49372a7-bb92-4daf-945a-ee03283fe26d-maas-api-xks-ca" pod "e49372a7-bb92-4daf-945a-ee03283fe26d" (UID: "e49372a7-bb92-4daf-945a-ee03283fe26d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:38:39.827024 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:39.826936    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e49372a7-bb92-4daf-945a-ee03283fe26d-kube-api-access-pdrkz" pod "e49372a7-bb92-4daf-945a-ee03283fe26d" (UID: "e49372a7-bb92-4daf-945a-ee03283fe26d"). InnerVolumeSpecName "kube-api-access-pdrkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:38:39.926071 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:39.925987    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e49372a7-bb92-4daf-945a-ee03283fe26d-maas-api-openshift-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:38:39.926071 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:39.926017    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e49372a7-bb92-4daf-945a-ee03283fe26d-maas-api-xks-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:38:39.926071 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:39.926026    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdrkz\" (UniqueName: \"kubernetes.io/projected/e49372a7-bb92-4daf-945a-ee03283fe26d-kube-api-access-pdrkz\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:38:39.958886 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:38:39.958845    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:38:40.604314 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:40.604283    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv"
Nov 02 02:38:40.604495 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:40.604290    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-2q8qv" event={"ID":"e49372a7-bb92-4daf-945a-ee03283fe26d","Type":"ContainerDied","Data":"70f721334c425fd91c997d812a40f622d0d8b5e027fec7dc6b0ccd501f052328"}
Nov 02 02:38:40.604495 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:40.604407    4515 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70f721334c425fd91c997d812a40f622d0d8b5e027fec7dc6b0ccd501f052328"
Nov 02 02:38:48.344838 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.344801    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457"]
Nov 02 02:38:48.345613 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.345287    4515 cpu_manager.go:414] "containerMap: removing container" podUID="e49372a7-bb92-4daf-945a-ee03283fe26d" containerName="revoke-keys"
Nov 02 02:38:48.345613 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.345307    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e49372a7-bb92-4daf-945a-ee03283fe26d" containerName="revoke-keys"
Nov 02 02:38:48.345613 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.345420    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="e49372a7-bb92-4daf-945a-ee03283fe26d" containerName="revoke-keys"
Nov 02 02:38:48.350063 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.350043    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457"
Nov 02 02:38:48.399923 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.399886    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9baf932a-4126-4a6e-8ae6-791fbfeaf140-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-g6457\" (UID: \"9baf932a-4126-4a6e-8ae6-791fbfeaf140\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457"
Nov 02 02:38:48.399923 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.399916    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46q8c\" (UniqueName: \"kubernetes.io/projected/9baf932a-4126-4a6e-8ae6-791fbfeaf140-kube-api-access-46q8c\") pod \"maas-api-revoke-sub-6b19338f-g6457\" (UID: \"9baf932a-4126-4a6e-8ae6-791fbfeaf140\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457"
Nov 02 02:38:48.400130 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.399938    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9baf932a-4126-4a6e-8ae6-791fbfeaf140-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-g6457\" (UID: \"9baf932a-4126-4a6e-8ae6-791fbfeaf140\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457"
Nov 02 02:38:48.403374 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.403193    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 02:38:48.452780 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.452755    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w4rht\""
Nov 02 02:38:48.452937 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.452804    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 02 02:38:48.452937 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.452853    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 02 02:38:48.454203 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.454184    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457"]
Nov 02 02:38:48.501329 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.501308    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9baf932a-4126-4a6e-8ae6-791fbfeaf140-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-g6457\" (UID: \"9baf932a-4126-4a6e-8ae6-791fbfeaf140\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457"
Nov 02 02:38:48.501478 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.501335    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46q8c\" (UniqueName: \"kubernetes.io/projected/9baf932a-4126-4a6e-8ae6-791fbfeaf140-kube-api-access-46q8c\") pod \"maas-api-revoke-sub-6b19338f-g6457\" (UID: \"9baf932a-4126-4a6e-8ae6-791fbfeaf140\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457"
Nov 02 02:38:48.501478 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.501356    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9baf932a-4126-4a6e-8ae6-791fbfeaf140-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-g6457\" (UID: \"9baf932a-4126-4a6e-8ae6-791fbfeaf140\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457"
Nov 02 02:38:48.501975 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.501951    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9baf932a-4126-4a6e-8ae6-791fbfeaf140-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-g6457\" (UID: \"9baf932a-4126-4a6e-8ae6-791fbfeaf140\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457"
Nov 02 02:38:48.503704 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.503686    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9baf932a-4126-4a6e-8ae6-791fbfeaf140-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-g6457\" (UID: \"9baf932a-4126-4a6e-8ae6-791fbfeaf140\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457"
Nov 02 02:38:48.562334 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.562311    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46q8c\" (UniqueName: \"kubernetes.io/projected/9baf932a-4126-4a6e-8ae6-791fbfeaf140-kube-api-access-46q8c\") pod \"maas-api-revoke-sub-6b19338f-g6457\" (UID: \"9baf932a-4126-4a6e-8ae6-791fbfeaf140\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457"
Nov 02 02:38:48.662333 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:48.662254    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457"
Nov 02 02:38:49.036170 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:49.036133    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457"]
Nov 02 02:38:49.039177 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:38:49.039147    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9baf932a_4126_4a6e_8ae6_791fbfeaf140.slice/crio-f2227efb695a83c7466bc6ff6bb530ff6d6fed9403a8ff783e644885bee8cf47 WatchSource:0}: Error finding container f2227efb695a83c7466bc6ff6bb530ff6d6fed9403a8ff783e644885bee8cf47: Status 404 returned error can't find the container with id f2227efb695a83c7466bc6ff6bb530ff6d6fed9403a8ff783e644885bee8cf47
Nov 02 02:38:49.642208 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:49.642176    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457" event={"ID":"9baf932a-4126-4a6e-8ae6-791fbfeaf140","Type":"ContainerStarted","Data":"6c798ef2a9689e4580ba5a93f12fee27333dbfa0650614c1c74e5463a60c93cc"}
Nov 02 02:38:49.642582 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:49.642225    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457" event={"ID":"9baf932a-4126-4a6e-8ae6-791fbfeaf140","Type":"ContainerStarted","Data":"f2227efb695a83c7466bc6ff6bb530ff6d6fed9403a8ff783e644885bee8cf47"}
Nov 02 02:38:49.759248 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:49.759192    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457" podStartSLOduration=1.759178344 podStartE2EDuration="1.759178344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:38:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:38:49.75816558 +0000 UTC m=+4018.970535389" watchObservedRunningTime="2026-11-02 02:38:49.759178344 +0000 UTC m=+4018.971548151"
Nov 02 02:38:50.648321 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:50.648288    4515 generic.go:356] "Generic (PLEG): container finished" podID="9baf932a-4126-4a6e-8ae6-791fbfeaf140" containerID="6c798ef2a9689e4580ba5a93f12fee27333dbfa0650614c1c74e5463a60c93cc" exitCode=0
Nov 02 02:38:50.648716 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:50.648356    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457" event={"ID":"9baf932a-4126-4a6e-8ae6-791fbfeaf140","Type":"ContainerDied","Data":"6c798ef2a9689e4580ba5a93f12fee27333dbfa0650614c1c74e5463a60c93cc"}
Nov 02 02:38:51.772235 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:51.772213    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457"
Nov 02 02:38:51.832575 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:51.832543    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9baf932a-4126-4a6e-8ae6-791fbfeaf140-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9baf932a-4126-4a6e-8ae6-791fbfeaf140-maas-api-openshift-ca\") pod \"9baf932a-4126-4a6e-8ae6-791fbfeaf140\" (UID: \"9baf932a-4126-4a6e-8ae6-791fbfeaf140\") "
Nov 02 02:38:51.832763 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:51.832617    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9baf932a-4126-4a6e-8ae6-791fbfeaf140-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9baf932a-4126-4a6e-8ae6-791fbfeaf140-maas-api-xks-ca\") pod \"9baf932a-4126-4a6e-8ae6-791fbfeaf140\" (UID: \"9baf932a-4126-4a6e-8ae6-791fbfeaf140\") "
Nov 02 02:38:51.832763 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:51.832657    4515 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9baf932a-4126-4a6e-8ae6-791fbfeaf140-kube-api-access-46q8c\" (UniqueName: \"kubernetes.io/projected/9baf932a-4126-4a6e-8ae6-791fbfeaf140-kube-api-access-46q8c\") pod \"9baf932a-4126-4a6e-8ae6-791fbfeaf140\" (UID: \"9baf932a-4126-4a6e-8ae6-791fbfeaf140\") "
Nov 02 02:38:51.832899 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:51.832879    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9baf932a-4126-4a6e-8ae6-791fbfeaf140-maas-api-openshift-ca" pod "9baf932a-4126-4a6e-8ae6-791fbfeaf140" (UID: "9baf932a-4126-4a6e-8ae6-791fbfeaf140"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:38:51.834728 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:51.834699    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9baf932a-4126-4a6e-8ae6-791fbfeaf140-kube-api-access-46q8c" pod "9baf932a-4126-4a6e-8ae6-791fbfeaf140" (UID: "9baf932a-4126-4a6e-8ae6-791fbfeaf140"). InnerVolumeSpecName "kube-api-access-46q8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:38:51.834840 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:51.834745    4515 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9baf932a-4126-4a6e-8ae6-791fbfeaf140-maas-api-xks-ca" pod "9baf932a-4126-4a6e-8ae6-791fbfeaf140" (UID: "9baf932a-4126-4a6e-8ae6-791fbfeaf140"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:38:51.933965 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:51.933889    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9baf932a-4126-4a6e-8ae6-791fbfeaf140-maas-api-openshift-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:38:51.933965 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:51.933915    4515 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9baf932a-4126-4a6e-8ae6-791fbfeaf140-maas-api-xks-ca\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:38:51.933965 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:51.933925    4515 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46q8c\" (UniqueName: \"kubernetes.io/projected/9baf932a-4126-4a6e-8ae6-791fbfeaf140-kube-api-access-46q8c\") on node \"ip-10-0-138-75.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:38:52.657674 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:52.657647    4515 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457"
Nov 02 02:38:52.657850 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:52.657644    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-g6457" event={"ID":"9baf932a-4126-4a6e-8ae6-791fbfeaf140","Type":"ContainerDied","Data":"f2227efb695a83c7466bc6ff6bb530ff6d6fed9403a8ff783e644885bee8cf47"}
Nov 02 02:38:52.657850 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:38:52.657776    4515 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2227efb695a83c7466bc6ff6bb530ff6d6fed9403a8ff783e644885bee8cf47"
Nov 02 02:39:09.958040 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:39:09.957997    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:39:12.612274 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:12.612244    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-gnn75_2cb9f581-50dc-4821-9ec3-3188bf13bc5a/manager/0.log"
Nov 02 02:39:13.961306 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:13.961275    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-qtvjg_6a91787f-612d-40cc-96ab-af1200465e9c/manager/0.log"
Nov 02 02:39:15.862248 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:15.862218    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0165d7d2-2q8qv_e49372a7-bb92-4daf-945a-ee03283fe26d/revoke-keys/0.log"
Nov 02 02:39:16.082549 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:16.082509    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6b19338f-g6457_9baf932a-4126-4a6e-8ae6-791fbfeaf140/revoke-keys/0.log"
Nov 02 02:39:16.518453 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:16.518427    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-c4kf9_8c2ae597-0a15-450a-8fc6-0b71488ba58a/revoke-keys/0.log"
Nov 02 02:39:21.704822 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:21.704785    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bf99ddbdb-jvtcn_909a5737-94eb-413e-87d7-9cd901ed51b2/kube-auth-proxy/0.log"
Nov 02 02:39:22.519026 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:22.518992    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-7wmnn_dd24d725-bfe3-4cde-9fc7-5d6d4e497995/istio-proxy/0.log"
Nov 02 02:39:23.723622 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:23.723590    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-gnmf7_612f51d4-bab6-4b28-b837-094c15c5166f/istio-proxy/0.log"
Nov 02 02:39:24.613105 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:24.613075    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-v8gx8_18653351-22cb-40db-ae44-1982a38f0a4c/istio-proxy/0.log"
Nov 02 02:39:28.433337 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:28.433307    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m_dab11213-84f0-4a27-bcd6-3f170312bba3/main/0.log"
Nov 02 02:39:39.424064 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:39.424029    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-gnn75_2cb9f581-50dc-4821-9ec3-3188bf13bc5a/manager/0.log"
Nov 02 02:39:39.958682 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:39:39.958645    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:39:40.486997 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:40.486968    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-qtvjg_6a91787f-612d-40cc-96ab-af1200465e9c/manager/0.log"
Nov 02 02:39:42.135240 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:42.135207    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0165d7d2-2q8qv_e49372a7-bb92-4daf-945a-ee03283fe26d/revoke-keys/0.log"
Nov 02 02:39:42.342060 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:42.342021    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6b19338f-g6457_9baf932a-4126-4a6e-8ae6-791fbfeaf140/revoke-keys/0.log"
Nov 02 02:39:42.562382 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:42.562353    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-c4kf9_8c2ae597-0a15-450a-8fc6-0b71488ba58a/revoke-keys/0.log"
Nov 02 02:39:46.981066 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:46.981027    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bf99ddbdb-jvtcn_909a5737-94eb-413e-87d7-9cd901ed51b2/kube-auth-proxy/0.log"
Nov 02 02:39:47.764422 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:47.764383    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-7wmnn_dd24d725-bfe3-4cde-9fc7-5d6d4e497995/istio-proxy/0.log"
Nov 02 02:39:48.970914 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:48.970883    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-gnmf7_612f51d4-bab6-4b28-b837-094c15c5166f/istio-proxy/0.log"
Nov 02 02:39:49.892527 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:49.892494    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-v8gx8_18653351-22cb-40db-ae44-1982a38f0a4c/istio-proxy/0.log"
Nov 02 02:39:53.435750 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:39:53.435722    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-j969m_dab11213-84f0-4a27-bcd6-3f170312bba3/main/0.log"
Nov 02 02:40:09.958182 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:40:09.958085    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:40:39.957620 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:40:39.957583    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:41:00.374412 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:00.374379    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cfrs6/must-gather-q9xdh"]
Nov 02 02:41:00.378097 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:00.374677    4515 cpu_manager.go:414] "containerMap: removing container" podUID="9baf932a-4126-4a6e-8ae6-791fbfeaf140" containerName="revoke-keys"
Nov 02 02:41:00.378097 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:00.374687    4515 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9baf932a-4126-4a6e-8ae6-791fbfeaf140" containerName="revoke-keys"
Nov 02 02:41:00.378097 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:00.374734    4515 memory_manager.go:365] "RemoveStaleState removing state" podUID="9baf932a-4126-4a6e-8ae6-791fbfeaf140" containerName="revoke-keys"
Nov 02 02:41:00.378982 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:00.378948    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cfrs6/must-gather-q9xdh"
Nov 02 02:41:00.438331 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:00.438293    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9dd4ca85-2243-4ff7-a222-156baa98113d-must-gather-output\") pod \"must-gather-q9xdh\" (UID: \"9dd4ca85-2243-4ff7-a222-156baa98113d\") " pod="openshift-must-gather-cfrs6/must-gather-q9xdh"
Nov 02 02:41:00.438513 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:00.438374    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mp6j\" (UniqueName: \"kubernetes.io/projected/9dd4ca85-2243-4ff7-a222-156baa98113d-kube-api-access-8mp6j\") pod \"must-gather-q9xdh\" (UID: \"9dd4ca85-2243-4ff7-a222-156baa98113d\") " pod="openshift-must-gather-cfrs6/must-gather-q9xdh"
Nov 02 02:41:00.482289 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:00.482252    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cfrs6\"/\"kube-root-ca.crt\""
Nov 02 02:41:00.482289 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:00.482276    4515 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cfrs6\"/\"openshift-service-ca.crt\""
Nov 02 02:41:00.482503 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:00.482276    4515 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cfrs6\"/\"default-dockercfg-ppkg9\""
Nov 02 02:41:00.483818 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:00.483790    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cfrs6/must-gather-q9xdh"]
Nov 02 02:41:00.539292 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:00.539252    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mp6j\" (UniqueName: \"kubernetes.io/projected/9dd4ca85-2243-4ff7-a222-156baa98113d-kube-api-access-8mp6j\") pod \"must-gather-q9xdh\" (UID: \"9dd4ca85-2243-4ff7-a222-156baa98113d\") " pod="openshift-must-gather-cfrs6/must-gather-q9xdh"
Nov 02 02:41:00.539493 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:00.539364    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9dd4ca85-2243-4ff7-a222-156baa98113d-must-gather-output\") pod \"must-gather-q9xdh\" (UID: \"9dd4ca85-2243-4ff7-a222-156baa98113d\") " pod="openshift-must-gather-cfrs6/must-gather-q9xdh"
Nov 02 02:41:00.539778 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:00.539755    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9dd4ca85-2243-4ff7-a222-156baa98113d-must-gather-output\") pod \"must-gather-q9xdh\" (UID: \"9dd4ca85-2243-4ff7-a222-156baa98113d\") " pod="openshift-must-gather-cfrs6/must-gather-q9xdh"
Nov 02 02:41:00.618992 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:00.618964    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mp6j\" (UniqueName: \"kubernetes.io/projected/9dd4ca85-2243-4ff7-a222-156baa98113d-kube-api-access-8mp6j\") pod \"must-gather-q9xdh\" (UID: \"9dd4ca85-2243-4ff7-a222-156baa98113d\") " pod="openshift-must-gather-cfrs6/must-gather-q9xdh"
Nov 02 02:41:00.693428 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:00.693398    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cfrs6/must-gather-q9xdh"
Nov 02 02:41:01.065617 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:01.065546    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cfrs6/must-gather-q9xdh"]
Nov 02 02:41:01.068879 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:41:01.068854    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9dd4ca85_2243_4ff7_a222_156baa98113d.slice/crio-7a5f33b4343c66c95817a805bc8fbf80583120f97b7985ce02de47aa347f8eb7 WatchSource:0}: Error finding container 7a5f33b4343c66c95817a805bc8fbf80583120f97b7985ce02de47aa347f8eb7: Status 404 returned error can't find the container with id 7a5f33b4343c66c95817a805bc8fbf80583120f97b7985ce02de47aa347f8eb7
Nov 02 02:41:01.070902 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:01.070886    4515 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 02:41:01.182949 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:01.182917    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cfrs6/must-gather-q9xdh" event={"ID":"9dd4ca85-2243-4ff7-a222-156baa98113d","Type":"ContainerStarted","Data":"7a5f33b4343c66c95817a805bc8fbf80583120f97b7985ce02de47aa347f8eb7"}
Nov 02 02:41:03.194756 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:03.194718    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cfrs6/must-gather-q9xdh" event={"ID":"9dd4ca85-2243-4ff7-a222-156baa98113d","Type":"ContainerStarted","Data":"dcf78e3053a3eecd84d9da89088df1c8eaa82634d2b8e877540e74125361fad6"}
Nov 02 02:41:03.194756 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:03.194761    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cfrs6/must-gather-q9xdh" event={"ID":"9dd4ca85-2243-4ff7-a222-156baa98113d","Type":"ContainerStarted","Data":"b1d4a49f1a0d8e60adac69f760075078544dcac7cc0d1480e6845783789e3b7a"}
Nov 02 02:41:03.314243 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:03.314190    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-cfrs6/must-gather-q9xdh" podStartSLOduration=2.117222739 podStartE2EDuration="3.314173987s" totalImagesPullingTime="1.196951248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:41:03.310815309 +0000 UTC m=+4152.523185119" watchObservedRunningTime="2026-11-02 02:41:03.314173987 +0000 UTC m=+4152.526543794"
Nov 02 02:41:09.752139 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:09.751300    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cqspb_696e3b37-d3bb-48e4-9836-afb85f9e34fd/global-pull-secret-syncer/0.log"
Nov 02 02:41:10.064224 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:41:10.063323    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"
Nov 02 02:41:13.295868 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:13.295835    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hbcss_0923847d-508c-4308-93f9-543d9bb3c251/konnectivity-agent/0.log"
Nov 02 02:41:14.142142 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:14.141473    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-75.us-west-1.compute.internal_e1f480d0264a553be3287c8f045c192e/haproxy/0.log"
Nov 02 02:41:19.133224 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.133177    4515 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"]
Nov 02 02:41:19.141989 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.141955    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:19.221184 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.221155    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b3dc069a-9d97-4398-92c1-97aa6cf7bfcb-podres\") pod \"perf-node-gather-daemonset-mqj8c\" (UID: \"b3dc069a-9d97-4398-92c1-97aa6cf7bfcb\") " pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:19.221184 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.221186    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b3dc069a-9d97-4398-92c1-97aa6cf7bfcb-proc\") pod \"perf-node-gather-daemonset-mqj8c\" (UID: \"b3dc069a-9d97-4398-92c1-97aa6cf7bfcb\") " pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:19.221414 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.221255    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3dc069a-9d97-4398-92c1-97aa6cf7bfcb-sys\") pod \"perf-node-gather-daemonset-mqj8c\" (UID: \"b3dc069a-9d97-4398-92c1-97aa6cf7bfcb\") " pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:19.221414 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.221297    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3dc069a-9d97-4398-92c1-97aa6cf7bfcb-lib-modules\") pod \"perf-node-gather-daemonset-mqj8c\" (UID: \"b3dc069a-9d97-4398-92c1-97aa6cf7bfcb\") " pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:19.221414 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.221323    4515 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8vfp\" (UniqueName: \"kubernetes.io/projected/b3dc069a-9d97-4398-92c1-97aa6cf7bfcb-kube-api-access-c8vfp\") pod \"perf-node-gather-daemonset-mqj8c\" (UID: \"b3dc069a-9d97-4398-92c1-97aa6cf7bfcb\") " pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:19.227214 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.227185    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"]
Nov 02 02:41:19.322333 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.322298    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b3dc069a-9d97-4398-92c1-97aa6cf7bfcb-podres\") pod \"perf-node-gather-daemonset-mqj8c\" (UID: \"b3dc069a-9d97-4398-92c1-97aa6cf7bfcb\") " pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:19.322333 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.322334    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b3dc069a-9d97-4398-92c1-97aa6cf7bfcb-proc\") pod \"perf-node-gather-daemonset-mqj8c\" (UID: \"b3dc069a-9d97-4398-92c1-97aa6cf7bfcb\") " pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:19.322615 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.322365    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3dc069a-9d97-4398-92c1-97aa6cf7bfcb-sys\") pod \"perf-node-gather-daemonset-mqj8c\" (UID: \"b3dc069a-9d97-4398-92c1-97aa6cf7bfcb\") " pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:19.322615 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.322402    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3dc069a-9d97-4398-92c1-97aa6cf7bfcb-lib-modules\") pod \"perf-node-gather-daemonset-mqj8c\" (UID: \"b3dc069a-9d97-4398-92c1-97aa6cf7bfcb\") " pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:19.322615 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.322433    4515 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8vfp\" (UniqueName: \"kubernetes.io/projected/b3dc069a-9d97-4398-92c1-97aa6cf7bfcb-kube-api-access-c8vfp\") pod \"perf-node-gather-daemonset-mqj8c\" (UID: \"b3dc069a-9d97-4398-92c1-97aa6cf7bfcb\") " pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:19.322615 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.322450    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b3dc069a-9d97-4398-92c1-97aa6cf7bfcb-proc\") pod \"perf-node-gather-daemonset-mqj8c\" (UID: \"b3dc069a-9d97-4398-92c1-97aa6cf7bfcb\") " pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:19.322615 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.322545    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b3dc069a-9d97-4398-92c1-97aa6cf7bfcb-podres\") pod \"perf-node-gather-daemonset-mqj8c\" (UID: \"b3dc069a-9d97-4398-92c1-97aa6cf7bfcb\") " pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:19.322615 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.322546    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3dc069a-9d97-4398-92c1-97aa6cf7bfcb-lib-modules\") pod \"perf-node-gather-daemonset-mqj8c\" (UID: \"b3dc069a-9d97-4398-92c1-97aa6cf7bfcb\") " pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:19.322833 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.322605    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3dc069a-9d97-4398-92c1-97aa6cf7bfcb-sys\") pod \"perf-node-gather-daemonset-mqj8c\" (UID: \"b3dc069a-9d97-4398-92c1-97aa6cf7bfcb\") " pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:19.383447 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.383381    4515 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8vfp\" (UniqueName: \"kubernetes.io/projected/b3dc069a-9d97-4398-92c1-97aa6cf7bfcb-kube-api-access-c8vfp\") pod \"perf-node-gather-daemonset-mqj8c\" (UID: \"b3dc069a-9d97-4398-92c1-97aa6cf7bfcb\") " pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:19.454867 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.454833    4515 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:19.826946 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:19.826915    4515 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"]
Nov 02 02:41:19.830357 ip-10-0-138-75 kubenswrapper[4515]: W1102 02:41:19.830327    4515 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb3dc069a_9d97_4398_92c1_97aa6cf7bfcb.slice/crio-cf89d2a49084ab7fd6065b56c330b6a0b74c435a48d57ef0a8026377572521dd WatchSource:0}: Error finding container cf89d2a49084ab7fd6065b56c330b6a0b74c435a48d57ef0a8026377572521dd: Status 404 returned error can't find the container with id cf89d2a49084ab7fd6065b56c330b6a0b74c435a48d57ef0a8026377572521dd
Nov 02 02:41:20.322568 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:20.322528    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c" event={"ID":"b3dc069a-9d97-4398-92c1-97aa6cf7bfcb","Type":"ContainerStarted","Data":"6c007d7f4b680d7d04c5c2d00095477f530388e462c307bec3e9be4a04fcc881"}
Nov 02 02:41:20.323067 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:20.322590    4515 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c" event={"ID":"b3dc069a-9d97-4398-92c1-97aa6cf7bfcb","Type":"ContainerStarted","Data":"cf89d2a49084ab7fd6065b56c330b6a0b74c435a48d57ef0a8026377572521dd"}
Nov 02 02:41:20.323067 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:20.322705    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:20.438993 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:20.438919    4515 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c" podStartSLOduration=1.438895974 podStartE2EDuration="1.438895974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 02:41:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:41:20.437300831 +0000 UTC m=+4169.649670633" watchObservedRunningTime="2026-11-02 02:41:20.438895974 +0000 UTC m=+4169.651265781"
Nov 02 02:41:26.337460 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:26.337432    4515 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cfrs6/perf-node-gather-daemonset-mqj8c"
Nov 02 02:41:29.218946 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:29.218842    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-77nlh_553be019-6080-45d6-bfba-348f682814c1/node-exporter/0.log"
Nov 02 02:41:29.631397 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:29.631313    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-77nlh_553be019-6080-45d6-bfba-348f682814c1/kube-rbac-proxy/0.log"
Nov 02 02:41:30.050733 ip-10-0-138-75 kubenswrapper[4515]: I1102 02:41:30.050701    4515 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-77nlh_553be019-6080-45d6-bfba-348f682814c1/init-textfile/0.log"
Nov 02 02:41:39.965992 ip-10-0-138-75 kubenswrapper[4515]: E1102 02:41:39.965902    4515 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-cnkn6" containerName="extractor"